-- Logs begin at Tue 2022-10-04 12:09:03 UTC, end at Tue 2022-10-04 12:50:43 UTC. -- Oct 04 12:33:40.743320 dv-cinder-860244-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 04 12:33:40.751346 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 04 12:33:40.753343 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 4 12:33:40 2022] *** Oct 04 12:33:40.753343 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 04 12:33:40.753343 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 04 12:33:40.753343 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: nodename: dv-cinder-860244-1-iscsi Oct 04 12:33:40.753343 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: machine: x86_64 Oct 04 12:33:40.753343 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: clock source: unix Oct 04 12:33:40.753726 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: pcre jit disabled Oct 04 12:33:40.753726 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: detected number of CPU cores: 4 Oct 04 12:33:40.753726 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: current working directory: / Oct 04 12:33:40.753726 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: detected binary path: /usr/bin/uwsgi-core Oct 04 12:33:40.753726 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: your processes number limit is 47680 Oct 04 12:33:40.753726 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: your memory page size is 4096 bytes Oct 04 12:33:40.753726 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: detected max file descriptor number: 2048 Oct 04 12:33:40.753726 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: lock engine: pthread robust mutexes Oct 04 12:33:40.753726 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: thunder lock: enabled Oct 04 12:33:40.753726 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 04 12:33:40.754365 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 04 12:33:40.794221 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: Python main interpreter initialized at 0x558e90fd29c0 Oct 04 12:33:40.794221 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: python threads support enabled Oct 04 12:33:40.794221 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: your server socket listen backlog is limited to 100 connections Oct 04 12:33:40.794221 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: your mercy for graceful operations on workers is 90 seconds Oct 04 12:33:40.794610 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: mapped 403077 bytes (393 KB) for 2 cores Oct 04 12:33:40.794691 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: *** Operational MODE: preforking *** Oct 04 12:33:40.794953 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: *** uWSGI is running in multiple interpreter mode *** Oct 04 12:33:40.794953 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: spawned uWSGI master process (pid: 93079) Oct 04 12:33:40.795158 dv-cinder-860244-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 04 12:33:40.796358 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: spawned uWSGI worker 1 (pid: 93091, cores: 1) Oct 04 12:33:40.796358 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: spawned uWSGI worker 2 (pid: 93092, cores: 1) Oct 04 12:33:40.796358 dv-cinder-860244-1-iscsi devstack@keystone.service[93079]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 04 12:33:45.087044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 12:33:45.087044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: from cryptography.utils import int_from_bytes Oct 04 12:33:45.087843 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 12:33:45.087843 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: from cryptography.utils import int_from_bytes Oct 04 12:33:45.088346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 12:33:45.088346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: from cryptography.utils import int_from_bytes Oct 04 12:33:45.088478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 12:33:45.088478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: from cryptography.utils import int_from_bytes Oct 04 12:33:45.138885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 12:33:45.139089 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 12:33:45.139250 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] command line args: [] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 12:33:45.139521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 12:33:45.139696 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ================================================================================ {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 12:33:45.139947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] admin_token = **** {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.140189 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] config_dir = [] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.140376 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.140557 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] config_source = [] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.140735 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] debug = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.140933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.142341 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.142341 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.142341 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.142341 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.142341 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] list_limit = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.142341 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] log_config_append = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.142726 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.142726 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] log_dir = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.142845 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] log_file = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.143016 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 12:33:45.143131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.143204 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 12:33:45.143274 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.143441 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] command line args: [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 12:33:45.143531 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.143650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 12:33:45.143743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.143817 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ================================================================================ {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 12:33:45.143898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.144325 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] admin_token = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.144425 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.144571 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.144658 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] config_dir = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.144759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.144876 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.144961 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] config_source = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.145033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.145103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] debug = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.145249 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.145353 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.145474 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.145569 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.145683 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.145775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.145934 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.146020 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.146200 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.146280 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.146428 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.146502 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.146623 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_config_append = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.146789 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.146950 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.147046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_dir = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.147127 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] publish_errors = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.147200 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_file = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.147354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.147494 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.147601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.147710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.147871 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.148037 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.148157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.148296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.148408 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.148518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.148518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.148770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] standard_threads = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.148884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.148993 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.149096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.149173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.149173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.149342 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.149431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] use_journal = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.149503 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.149578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] use_json = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.149578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] use_stderr = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.149749 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.149826 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] use_syslog = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.149916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.149993 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.150104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.150183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.150254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.150354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.150504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.150611 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.150850 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.150997 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.151082 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.151181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] publish_errors = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.151325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.151406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.151555 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.151633 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.151789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.151873 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.152033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.152122 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.152809 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.152890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.153048 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] standard_threads = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.153124 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.153281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.153359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.153523 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.153600 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.153700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.153778 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.153877 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] use_journal = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.153990 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.154087 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] use_json = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.154238 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] use_stderr = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.154317 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.154447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] use_syslog = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.154576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.154651 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 12:33:45.154726 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.154878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.154961 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.155119 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.155207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.155379 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.155479 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.155621 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.155727 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.155870 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.155958 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.156110 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.156191 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.156359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.156438 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.156590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.156674 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.156823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.156901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.157054 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.157137 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.157291 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.157453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.157529 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.157598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.157764 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.157842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.157995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.158091 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.158251 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.158329 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.158497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.158585 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.158733 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.158810 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.158967 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.159043 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.159213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.159296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.159445 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.159528 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.159678 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.159755 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.159914 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.159991 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.160144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.160224 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.160373 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.160456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.160601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.160684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.160832 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.160911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.161067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.161145 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.161310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.161385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.161539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.161616 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.161775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.161852 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.162005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.162112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.162262 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.162341 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.162497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.162572 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.162742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.162818 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.162962 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.163042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.163195 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.163273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.163432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.163510 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.163665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.163744 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.163899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.163974 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.164131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.164203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.164360 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.164433 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.164593 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.164668 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.164825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.164897 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.165055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.165185 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.165291 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.165537 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.165537 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.165745 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.165899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.166014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.166014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.166332 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.166332 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.166573 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.166693 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.166840 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.166949 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.167085 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.167193 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.167303 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.167449 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.167558 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.167666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.167819 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.168562 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.168683 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.168824 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.168944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.169082 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.169191 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.169299 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.169435 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.169550 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.169671 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.169811 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.169918 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.170070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.170180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.170260 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.170377 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.170455 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.170527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.170649 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.170723 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.170794 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.170877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.170993 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.171069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.171140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.171210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.171280 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.171358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.171437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.171509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.171586 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.171692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.171810 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.171896 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.172051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.172143 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.172286 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.172370 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.172525 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.172614 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.172761 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.172844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.173014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.173098 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.173229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.173316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.173460 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.173543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.173689 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.173779 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.173919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.174005 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.174177 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.174260 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.174407 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.174485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.174642 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.174801 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.174877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.174946 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.175105 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.175188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.175338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.175428 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.175593 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.175735 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.175821 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.175892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.176068 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.176155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.176296 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] auth.external = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.176420 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.176525 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.176604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.176767 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.176853 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.177010 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.177090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.177248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.177332 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] auth.password = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.177475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] auth.token = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.177552 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.177706 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.177785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.177939 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.178019 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.178192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.178304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.178430 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.178511 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.178664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.178744 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.178912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.179033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.179136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.179214 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.179370 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] credential.caching = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.179450 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.179609 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.179686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.179856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.179938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.180104 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.180187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.180344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.180422 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.180576 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.external = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.180653 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.180808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.180886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.181052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.181129 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.181303 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.181384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.181540 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.181619 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.password = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.181773 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.token = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.181851 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.182015 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.182131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.182272 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.182352 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.182522 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.182599 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.182754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.182831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.183004 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.183084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.183251 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.183329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] federation.caching = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.183998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.184095 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.184240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.184412 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.184491 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.184664 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.184741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.184897 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.184983 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.185147 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.185226 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.185384 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.185462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.185638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.185724 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.185875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.185952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.186156 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.186238 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.186394 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.186479 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.186640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.186718 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.186874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.caching = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.186951 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.187123 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.187202 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.187368 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.187446 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.187603 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.187680 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.187843 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.187921 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.188088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.188170 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.188319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.188396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.188555 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.188634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.188805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.188945 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.189068 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.189191 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.189390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.189511 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.189629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.189747 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.189864 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.189982 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.190168 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.190290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.190410 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.190528 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.190646 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.190763 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.190889 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.191009 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.191126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.191243 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.191367 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.191487 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.191613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.191740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.191861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.191979 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.192126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.192246 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.192373 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.192500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.192626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.192746 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.192863 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.192981 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.193107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.193226 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.193343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.193461 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.193579 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.193703 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.193831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.193950 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.194092 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.194214 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.194331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.194448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.194576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.194715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.194835 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.194952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.195069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.195186 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.195312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.195431 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.195549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.195668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.195785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.195902 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.196028 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.196147 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.196264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.196381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.196499 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.196625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.196745 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.196862 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.196979 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.197097 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.197220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.197347 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.197465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.197582 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.197701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.197818 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.197935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.198093 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.198220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.198338 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.198456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.198574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.198691 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.198818 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.198937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.199065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.199758 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.199878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.199996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.200123 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.200245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.200363 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.200489 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.200609 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.200727 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.200855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.200974 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.201092 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.201216 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.201383 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.201504 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.201629 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.201749 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.201867 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.201985 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.202128 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.202247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.202377 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.202498 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.202622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.202740 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.202858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.202976 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.203102 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.203221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.203338 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.203455 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.203572 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.203690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.203806 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.203907 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.204006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.204123 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.204241 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.204358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.204485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.204605 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.204723 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.204830 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.204946 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.205065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.205065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.205301 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.205419 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.205536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.205653 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.205771 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.205897 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.206016 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.206156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.206275 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.206275 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.206495 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.206495 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.206730 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.206848 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.206965 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.206965 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.207185 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.207185 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.207415 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.207539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.207658 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.207775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.207892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.208018 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.208137 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.208255 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.208372 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.208489 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.208606 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.208767 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] resource.caching = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.208890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.209014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.209133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.209252 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.209369 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.209495 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.209615 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.209736 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.209854 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.209971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.210102 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.210235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.210355 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.210473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.210591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.210709 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.210826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.210953 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.211072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.211189 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] role.caching = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.211308 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.211425 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] role.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.211543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.211669 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.211788 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.211905 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.212023 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.212141 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.212268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.212389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.212508 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.212625 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.212743 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.212861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.213056 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.213178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.213295 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.213413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.213531 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.213648 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.213774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.213893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.214011 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.214183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.214302 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.214419 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.214551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.214691 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.214814 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.214921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.215037 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.215037 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.215264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.215383 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.215502 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.215619 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] role.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.215737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.215854 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] role.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.215980 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.216098 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.216222 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.216222 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.216443 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.216561 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.216687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.216806 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.216930 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.217049 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.217166 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.217289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.217417 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.217537 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.217655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.217773 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.217890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.218007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.218157 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.218278 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.218395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.218513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.218631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.218748 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.218874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.218993 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.219110 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.219227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.219344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.219469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.219469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] token.caching = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.219695 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.219813 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.219930 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.220047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.220172 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.220291 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.220409 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.220526 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.220643 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.220761 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.220886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.221005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.221122 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.221240 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.221357 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.221474 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.221600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.221719 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.221838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.221954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.222088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.222208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.222337 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.222457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.222574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.222691 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.222807 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.222925 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.223050 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.223170 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.223296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.223422 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.223541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.223658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.223774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.223900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.connection = **** {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.224019 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.224137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.224254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.224371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.224488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.224613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.224731 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.224848 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.224966 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.225082 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.225200 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.225325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.225444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.225561 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.225678 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.225795 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.225912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.226058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.226180 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.226297 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.226297 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.226517 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.226634 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.226763 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.226884 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.227001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.227118 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.227236 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.227353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.227479 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.227598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.227715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.227916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.228038 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.228167 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.228295 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.228415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 12:33:45.228543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.228543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.connection = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.228543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.228543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.228543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.228543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.228543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.228543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.229402 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.229402 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.229402 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.229402 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.229875 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.229875 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.230224 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.230857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.231039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.231276 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.231520 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.231759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 12:33:45.231932 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 12:33:45.256781 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.257013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.257238 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.257909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.258156 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.258387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.266075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.266764 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.294940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.295169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.295794 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.296035 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.349347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.349529 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.399195 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.399423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.399631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.399872 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.558461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.559152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.559236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.559315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.559414 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.559542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.559627 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.559761 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.559844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.559979 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.560068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.560194 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.584179 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.584374 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.584459 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.584556 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.584635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.584709 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.584858 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.584941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.585080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.585165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.585294 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.585377 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.585507 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.585598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.597168 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.597310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.610961 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.611135 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.611220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.611363 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.611447 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.611590 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.611673 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.611814 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.611905 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.612032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.612115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.612243 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.612328 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.612454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 12:33:45.629702 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.629912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 12:33:45.629996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.630201 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 12:33:45.630287 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.630474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 12:33:45.630557 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.630734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 12:33:45.630816 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.631007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.631097 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.631272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.631353 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.631532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.631615 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.631794 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.641133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.641303 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.641415 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.641584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.641694 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.641857 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.641982 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.642176 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.649220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.649323 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.652134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.652748 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.652956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.653039 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.653244 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.653326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.653531 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.653615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.653820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.653903 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.654106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.654204 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.654383 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.654465 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.654657 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.654739 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.654931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.655014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.655204 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.655284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.655471 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.655550 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.678361 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.678491 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.679825 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.679955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.680112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.680246 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.684764 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.684923 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.685050 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.685215 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.685338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.685504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.691626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.691852 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.691936 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.692147 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.697694 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.697972 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.698075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.698298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.698380 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.698590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.698675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.698874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.699000 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.699232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.699334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.699473 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.699556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.699749 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.699829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.700025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.715646 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.715923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.716006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.716214 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.719774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.720072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.723011 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.723319 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.723403 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.723621 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.727486 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.727876 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.731171 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.731529 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.734391 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.734694 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.734776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.734978 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.735060 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.735261 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.735352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.735543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.735629 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.735842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.735935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.736176 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.751439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.751728 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.751923 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.752007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.752217 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.752319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.752488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.752754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.759020 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.759574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.760564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.761125 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.762198 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.762479 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.762831 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.763113 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.765126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.765419 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.765706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.765788 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.765994 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.766111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.766334 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.766425 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.766619 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.767026 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.778163 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.778452 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.778736 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.778916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.779022 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.779203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.779410 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.779500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.779706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.779792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.779998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.780127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.780429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.780718 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.795675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.796534 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.798588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.799471 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.801447 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.802429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.802893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.803876 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.804372 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.804655 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.805057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.805344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.805631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.806122 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.811477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.812505 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.812973 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.814030 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.815938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.816366 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.816653 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.816930 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.817065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.817412 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.817707 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.817992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.825005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.825312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.825600 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.826089 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.826409 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.826700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.832661 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.832968 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.833266 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.833547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.833694 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 12:33:45.833830 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.834013 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.834178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.834337 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.834424 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.834619 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.834700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.834897 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.834989 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.835178 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.835260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.835458 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.835541 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.835724 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.835812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.835995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.836085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.836289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.836381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.836622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.836894 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.837166 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.837447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 12:33:45.871965 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558e90fd29c0 pid: 93092 (default app) Oct 04 12:33:45.872939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558e90fd29c0 pid: 93091 (default app) Oct 04 12:33:45.875537 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05adb86-ace5-44db-8699-1d468d0ccf53 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:45.875824 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05adb86-ace5-44db-8699-1d468d0ccf53 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:45.876096 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05adb86-ace5-44db-8699-1d468d0ccf53 None None] PATH_INFO: `/v3` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:45.880584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Tue Oct 4 12:33:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:33:52.937919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e848b-be36-401f-9d15-7b8376b2e447 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:52.938487 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e848b-be36-401f-9d15-7b8376b2e447 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:52.938487 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e848b-be36-401f-9d15-7b8376b2e447 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:52.942697 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:33:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:33:52.948471 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6fa10-5135-4d8b-b6bd-ca08331ad53c None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:52.948952 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6fa10-5135-4d8b-b6bd-ca08331ad53c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:52.949094 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6fa10-5135-4d8b-b6bd-ca08331ad53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:53.019682 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: INFO dbcounter [None req-d5c6fa10-5135-4d8b-b6bd-ca08331ad53c None None] Registered counter for database keystone Oct 04 12:33:53.038986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG oslo_db.sqlalchemy.engines [None req-d5c6fa10-5135-4d8b-b6bd-ca08331ad53c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93092) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 12:33:53.045977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writer thread running {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 12:33:53.227387 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG passlib.handlers.bcrypt [None req-d5c6fa10-5135-4d8b-b6bd-ca08331ad53c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93092) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 12:33:53.227685 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG passlib.handlers.bcrypt [None req-d5c6fa10-5135-4d8b-b6bd-ca08331ad53c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93092) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 12:33:53.287309 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-d5c6fa10-5135-4d8b-b6bd-ca08331ad53c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:33:53.288945 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-d5c6fa10-5135-4d8b-b6bd-ca08331ad53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:33:53.422388 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:33:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 04 12:33:53.732603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e848b-be36-401f-9d15-7b8376b2e447 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:33:53.802882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: INFO dbcounter [None req-646e848b-be36-401f-9d15-7b8376b2e447 None None] Registered counter for database keystone Oct 04 12:33:53.824878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG oslo_db.sqlalchemy.engines [None req-646e848b-be36-401f-9d15-7b8376b2e447 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93091) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 12:33:53.831255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writer thread running {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 12:33:53.954395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e848b-be36-401f-9d15-7b8376b2e447 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:33:53.958790 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae902a1-3464-4395-bf03-459f46fc63f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:33:53.959423 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae902a1-3464-4395-bf03-459f46fc63f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:53.959706 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae902a1-3464-4395-bf03-459f46fc63f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:53.959979 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae902a1-3464-4395-bf03-459f46fc63f3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:53.968508 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae902a1-3464-4395-bf03-459f46fc63f3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:33:54.022027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4ae902a1-3464-4395-bf03-459f46fc63f3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 12:33:54.022392 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae902a1-3464-4395-bf03-459f46fc63f3 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:33:54.032988 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Tue Oct 4 12:33:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 12:33:54.038956 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6fa10-5135-4d8b-b6bd-ca08331ad53c None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:33:54.056250 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6fa10-5135-4d8b-b6bd-ca08331ad53c None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:33:54.061435 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf04b0-1be7-49ac-b42d-95ae276d008d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:33:54.062846 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf04b0-1be7-49ac-b42d-95ae276d008d admin admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:54.063156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf04b0-1be7-49ac-b42d-95ae276d008d admin admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:54.063441 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf04b0-1be7-49ac-b42d-95ae276d008d admin admin] PATH_INFO: `/v3/projects/f85b0de3065741e781a91cbb56d099f5` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:54.065269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf04b0-1be7-49ac-b42d-95ae276d008d admin admin] RBAC: Authorizing `identity:get_project(project_id=f85b0de3065741e781a91cbb56d099f5)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:33:54.119856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: WARNING keystone.common.rbac_enforcer.enforcer [None req-75cf04b0-1be7-49ac-b42d-95ae276d008d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 12:33:54.120205 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf04b0-1be7-49ac-b42d-95ae276d008d admin admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:33:54.124839 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Tue Oct 4 12:33:54 2022] GET /identity/v3/projects/f85b0de3065741e781a91cbb56d099f5 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:33:56.276547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2af88-61ed-4c1c-8dcf-0cc8c01060fa None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:56.277000 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2af88-61ed-4c1c-8dcf-0cc8c01060fa None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:56.277177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2af88-61ed-4c1c-8dcf-0cc8c01060fa None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:56.281161 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:33:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:33:56.286988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd058b5-caa4-4bac-883e-fb2641f447be None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:56.287363 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd058b5-caa4-4bac-883e-fb2641f447be None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:56.287529 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd058b5-caa4-4bac-883e-fb2641f447be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:56.307136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-5fd058b5-caa4-4bac-883e-fb2641f447be None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:33:56.309145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-5fd058b5-caa4-4bac-883e-fb2641f447be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:33:56.317037 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:33:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 04 12:33:56.593236 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2af88-61ed-4c1c-8dcf-0cc8c01060fa None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:33:56.605189 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2af88-61ed-4c1c-8dcf-0cc8c01060fa None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:33:56.609344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57befc-4b51-4057-801f-c96314d913c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:33:56.609935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57befc-4b51-4057-801f-c96314d913c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:56.610241 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57befc-4b51-4057-801f-c96314d913c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:56.610515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57befc-4b51-4057-801f-c96314d913c3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:56.618110 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57befc-4b51-4057-801f-c96314d913c3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:33:56.619887 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff57befc-4b51-4057-801f-c96314d913c3 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:33:56.628508 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Tue Oct 4 12:33:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 12:33:56.634046 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd058b5-caa4-4bac-883e-fb2641f447be None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:33:56.643643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd058b5-caa4-4bac-883e-fb2641f447be None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:33:56.648308 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace2659-6b9a-49cf-9371-11ec428cc730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:33:56.648936 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace2659-6b9a-49cf-9371-11ec428cc730 admin admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:56.649213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace2659-6b9a-49cf-9371-11ec428cc730 admin admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:56.649483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace2659-6b9a-49cf-9371-11ec428cc730 admin admin] PATH_INFO: `/v3/users/ba3be3b6260847d081e4ed3e9d15d818` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:56.650808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace2659-6b9a-49cf-9371-11ec428cc730 admin admin] RBAC: Authorizing `identity:get_user(user_id=ba3be3b6260847d081e4ed3e9d15d818)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:33:56.653291 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace2659-6b9a-49cf-9371-11ec428cc730 admin admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:33:56.654846 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Tue Oct 4 12:33:56 2022] GET /identity/v3/users/ba3be3b6260847d081e4ed3e9d15d818 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:33:58.882193 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a966eda-c0dd-4e2f-9da1-20b05514340a None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:58.882764 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a966eda-c0dd-4e2f-9da1-20b05514340a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:58.882764 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a966eda-c0dd-4e2f-9da1-20b05514340a None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:58.884837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:33:58.889483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384246ab-b37c-4975-a7af-d25416a9e4b6 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:58.890114 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384246ab-b37c-4975-a7af-d25416a9e4b6 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:58.890114 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384246ab-b37c-4975-a7af-d25416a9e4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:58.909049 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-384246ab-b37c-4975-a7af-d25416a9e4b6 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:33:58.910257 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-384246ab-b37c-4975-a7af-d25416a9e4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:33:58.933397 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bcd94-9970-442e-85af-d7f0938e525a None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:58.934407 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bcd94-9970-442e-85af-d7f0938e525a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:58.934752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bcd94-9970-442e-85af-d7f0938e525a None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:58.936475 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:33:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:33:58.942055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd93c15-5d9d-44a7-b574-358623a98e9e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:58.942346 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd93c15-5d9d-44a7-b574-358623a98e9e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:58.942673 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd93c15-5d9d-44a7-b574-358623a98e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:58.987230 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG passlib.handlers.bcrypt [None req-5dd93c15-5d9d-44a7-b574-358623a98e9e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93091) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 12:33:58.987622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG passlib.handlers.bcrypt [None req-5dd93c15-5d9d-44a7-b574-358623a98e9e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93091) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 12:33:59.004125 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:33:59.037138 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5dd93c15-5d9d-44a7-b574-358623a98e9e None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:33:59.038790 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5dd93c15-5d9d-44a7-b574-358623a98e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:33:59.143265 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:33:59.276063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384246ab-b37c-4975-a7af-d25416a9e4b6 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:33:59.303920 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:33:59.393997 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384246ab-b37c-4975-a7af-d25416a9e4b6 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:33:59.410110 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd93c15-5d9d-44a7-b574-358623a98e9e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:33:59.434593 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384115eb-65a3-4876-9548-cc148e8bfcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:33:59.435266 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384115eb-65a3-4876-9548-cc148e8bfcd6 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:59.435550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384115eb-65a3-4876-9548-cc148e8bfcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:59.435825 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384115eb-65a3-4876-9548-cc148e8bfcd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:59.455090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384115eb-65a3-4876-9548-cc148e8bfcd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-384115eb-65a3-4876-9548-cc148e8bfcd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:33:59.468272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:33:59.469970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:33:59.471086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:33:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:33:59.475479 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384115eb-65a3-4876-9548-cc148e8bfcd6 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:33:59.528702 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd93c15-5d9d-44a7-b574-358623a98e9e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:33:59.531363 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:33:59.573654 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecadd67-df2e-4430-ad6e-39b0d144fa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:33:59.574711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:33:59.575120 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecadd67-df2e-4430-ad6e-39b0d144fa7d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:59.575403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecadd67-df2e-4430-ad6e-39b0d144fa7d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:59.575671 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecadd67-df2e-4430-ad6e-39b0d144fa7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:59.600139 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384115eb-65a3-4876-9548-cc148e8bfcd6 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:33:59.641005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2331106-2474-4b40-bfc2-531499b2ebe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:33:59.641638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2331106-2474-4b40-bfc2-531499b2ebe9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:59.641915 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2331106-2474-4b40-bfc2-531499b2ebe9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:59.642215 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2331106-2474-4b40-bfc2-531499b2ebe9 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:59.642898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2331106-2474-4b40-bfc2-531499b2ebe9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:33:59.643198 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2331106-2474-4b40-bfc2-531499b2ebe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:33:59.644962 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2331106-2474-4b40-bfc2-531499b2ebe9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:33:59.658573 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:33:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:33:59.660473 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecadd67-df2e-4430-ad6e-39b0d144fa7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:33:59.662127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecadd67-df2e-4430-ad6e-39b0d144fa7d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:33:59.664473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2331106-2474-4b40-bfc2-531499b2ebe9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:33:59.806418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:33:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 12:33:59.812388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecadd67-df2e-4430-ad6e-39b0d144fa7d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:33:59.813930 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2331106-2474-4b40-bfc2-531499b2ebe9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:33:59.815725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:33:59.856920 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75f5a2-bd58-459b-9a5a-eb8eaec43900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:33:59.857565 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75f5a2-bd58-459b-9a5a-eb8eaec43900 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:59.857844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75f5a2-bd58-459b-9a5a-eb8eaec43900 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:59.858149 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75f5a2-bd58-459b-9a5a-eb8eaec43900 None admin] PATH_INFO: `/v3/users/ba3be3b6260847d081e4ed3e9d15d818` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:59.859856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75f5a2-bd58-459b-9a5a-eb8eaec43900 None admin] RBAC: Authorizing `identity:get_user(user_id=ba3be3b6260847d081e4ed3e9d15d818)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:33:59.861005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75f5a2-bd58-459b-9a5a-eb8eaec43900 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:33:59.862702 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:33:59 2022] GET /identity/v3/users/ba3be3b6260847d081e4ed3e9d15d818 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:33:59.871604 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75f5a2-bd58-459b-9a5a-eb8eaec43900 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:33:59.889667 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:33:59.946315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecadd67-df2e-4430-ad6e-39b0d144fa7d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:33:59.989936 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dab519-0144-4b9c-8b3e-b6788a30a173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:33:59.990812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dab519-0144-4b9c-8b3e-b6788a30a173 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:33:59.991089 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dab519-0144-4b9c-8b3e-b6788a30a173 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:33:59.991361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dab519-0144-4b9c-8b3e-b6788a30a173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:33:59.999878 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75f5a2-bd58-459b-9a5a-eb8eaec43900 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:00.011349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dab519-0144-4b9c-8b3e-b6788a30a173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:00.013450 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dab519-0144-4b9c-8b3e-b6788a30a173 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-51dab519-0144-4b9c-8b3e-b6788a30a173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:34:00.033495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:00.035289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:34:00.036885 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:34:00.036885 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:34:00.036885 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:00.036885 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:34:00.040989 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dab519-0144-4b9c-8b3e-b6788a30a173 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:00.064152 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a35f85-9c46-4fd5-ba4b-51c3c472bf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:00.065173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a35f85-9c46-4fd5-ba4b-51c3c472bf08 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:00.065656 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a35f85-9c46-4fd5-ba4b-51c3c472bf08 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:00.066140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a35f85-9c46-4fd5-ba4b-51c3c472bf08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:00.067992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a35f85-9c46-4fd5-ba4b-51c3c472bf08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:00.070595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a35f85-9c46-4fd5-ba4b-51c3c472bf08 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:00.079136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:33:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:00.081205 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a35f85-9c46-4fd5-ba4b-51c3c472bf08 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:00.103273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:00.155342 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dab519-0144-4b9c-8b3e-b6788a30a173 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:00.185397 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:00.206437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05f53f-0d0d-4575-9178-d0de50eee8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:00.207024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05f53f-0d0d-4575-9178-d0de50eee8fb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:00.207401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05f53f-0d0d-4575-9178-d0de50eee8fb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:00.207795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05f53f-0d0d-4575-9178-d0de50eee8fb None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:00.208580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05f53f-0d0d-4575-9178-d0de50eee8fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:00.208992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05f53f-0d0d-4575-9178-d0de50eee8fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:00.210831 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05f53f-0d0d-4575-9178-d0de50eee8fb None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:00.213954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a35f85-9c46-4fd5-ba4b-51c3c472bf08 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:00.232442 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 10/20] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:00.262686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d64a6d-022d-4140-aff5-3519361ae356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:00.263660 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d64a6d-022d-4140-aff5-3519361ae356 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:00.264088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d64a6d-022d-4140-aff5-3519361ae356 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:00.264513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d64a6d-022d-4140-aff5-3519361ae356 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:00.265686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d64a6d-022d-4140-aff5-3519361ae356 None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.domain.id=default, user.id=ba3be3b6260847d081e4ed3e9d15d818) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:00.266197 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d64a6d-022d-4140-aff5-3519361ae356 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:00.268616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d64a6d-022d-4140-aff5-3519361ae356 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:00.290845 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 11/21] 10.222.0.33 () {56 vars in 1322 bytes} [Tue Oct 4 12:34:00 2022] GET /identity/v3/role_assignments?user.id=ba3be3b6260847d081e4ed3e9d15d818&scope.domain.id=default&role.id=fc68d072459d4720b9666c93bd535d63 => generated 233 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:04.131356 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7179b-a637-4baa-8f60-ccebd7a5f064 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:04.132398 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7179b-a637-4baa-8f60-ccebd7a5f064 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:04.132916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7179b-a637-4baa-8f60-ccebd7a5f064 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:04.134590 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 11/22] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:04.140784 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf6954-4c42-4809-937d-56be309959d6 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:04.141743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf6954-4c42-4809-937d-56be309959d6 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:04.142248 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf6954-4c42-4809-937d-56be309959d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:04.151380 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5948c8-781f-4071-8e0d-9e88c656cc73 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:04.152032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5948c8-781f-4071-8e0d-9e88c656cc73 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:04.152635 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5948c8-781f-4071-8e0d-9e88c656cc73 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:04.154261 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 12/23] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:34:04.162191 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df34ee3-362d-4e09-80e5-46ce5963473c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:04.162816 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df34ee3-362d-4e09-80e5-46ce5963473c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:04.163311 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df34ee3-362d-4e09-80e5-46ce5963473c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:04.203218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-0df34ee3-362d-4e09-80e5-46ce5963473c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:04.203430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-74bf6954-4c42-4809-937d-56be309959d6 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:04.204476 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-74bf6954-4c42-4809-937d-56be309959d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:04.205161 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0df34ee3-362d-4e09-80e5-46ce5963473c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:04.363651 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 12/24] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:34:04.401929 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 13/25] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:34:04.501095 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ff56b-91ee-4d28-b8a7-0f02e08d3d8e None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:04.501583 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ff56b-91ee-4d28-b8a7-0f02e08d3d8e None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:04.502096 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ff56b-91ee-4d28-b8a7-0f02e08d3d8e None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:04.505264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 13/26] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:04.511847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fcbfe-a161-43ef-be50-7786192bcfcb None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:04.512322 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fcbfe-a161-43ef-be50-7786192bcfcb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:04.512804 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fcbfe-a161-43ef-be50-7786192bcfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:04.559003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-278fcbfe-a161-43ef-be50-7786192bcfcb None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:04.560992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-278fcbfe-a161-43ef-be50-7786192bcfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:04.667576 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:04.721667 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 14/27] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:04.743867 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37902c-6832-4e02-a28f-eed6c978935a None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:04.744340 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37902c-6832-4e02-a28f-eed6c978935a None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:04.744832 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37902c-6832-4e02-a28f-eed6c978935a None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:04.747468 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 14/28] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:04.753736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd641c8a-256e-4052-85e6-9fd87f0b78aa None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:04.754024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd641c8a-256e-4052-85e6-9fd87f0b78aa None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:04.754605 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd641c8a-256e-4052-85e6-9fd87f0b78aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:04.769303 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37902c-6832-4e02-a28f-eed6c978935a None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:04.794290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-dd641c8a-256e-4052-85e6-9fd87f0b78aa None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:04.796386 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dd641c8a-256e-4052-85e6-9fd87f0b78aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:04.956853 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:05.004745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 15/29] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:05.048694 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37902c-6832-4e02-a28f-eed6c978935a None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:05.093247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd641c8a-256e-4052-85e6-9fd87f0b78aa None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:05.143656 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dba9d1-990c-44c8-9dbc-7b93dde229c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:05.143656 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dba9d1-990c-44c8-9dbc-7b93dde229c2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:05.143656 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dba9d1-990c-44c8-9dbc-7b93dde229c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:05.143656 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dba9d1-990c-44c8-9dbc-7b93dde229c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:05.179435 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dba9d1-990c-44c8-9dbc-7b93dde229c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:05.182276 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dba9d1-990c-44c8-9dbc-7b93dde229c2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-c0dba9d1-990c-44c8-9dbc-7b93dde229c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:34:05.226866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:05.228483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:34:05.230015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:34:05.230015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:34:05.230015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:05.230015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 15/30] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:05.231184 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dba9d1-990c-44c8-9dbc-7b93dde229c2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:05.336537 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd641c8a-256e-4052-85e6-9fd87f0b78aa None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:05.374270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:05.467225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b3e8a-3ec3-4998-96ba-ef8fb3bb2cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:05.468251 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b3e8a-3ec3-4998-96ba-ef8fb3bb2cdd None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:05.468745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b3e8a-3ec3-4998-96ba-ef8fb3bb2cdd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:05.469213 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b3e8a-3ec3-4998-96ba-ef8fb3bb2cdd None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:05.470788 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b3e8a-3ec3-4998-96ba-ef8fb3bb2cdd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:05.473435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b3e8a-3ec3-4998-96ba-ef8fb3bb2cdd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:05.560637 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dba9d1-990c-44c8-9dbc-7b93dde229c2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:05.658585 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:05.675222 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 16/31] 10.222.0.33 () {60 vars in 1131 bytes} [Tue Oct 4 12:34:05 2022] POST /identity/v3/roles => generated 219 bytes in 582 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:34:05.676492 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b3e8a-3ec3-4998-96ba-ef8fb3bb2cdd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:05.677093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7342a-52f7-4890-be08-05dc5f685e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:05.678364 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7342a-52f7-4890-be08-05dc5f685e78 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:05.679122 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7342a-52f7-4890-be08-05dc5f685e78 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:05.679777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7342a-52f7-4890-be08-05dc5f685e78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:05.708525 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7342a-52f7-4890-be08-05dc5f685e78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-dec7342a-52f7-4890-be08-05dc5f685e78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:05.721207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:05.723135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:05.724999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:05.724999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:05.724999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:05.724999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:05.724999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:05.724999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:05.724999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:05.724999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:05.724999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:05.724999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:05.724999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:05.724999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:05.728847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 16/32] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:34:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 499 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:05.731080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7342a-52f7-4890-be08-05dc5f685e78 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:05.903991 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b3e8a-3ec3-4998-96ba-ef8fb3bb2cdd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:05.957794 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7342a-52f7-4890-be08-05dc5f685e78 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:05.963798 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754b820-d05e-493c-a5d9-462e957f1de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:05.964436 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754b820-d05e-493c-a5d9-462e957f1de4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:05.964723 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754b820-d05e-493c-a5d9-462e957f1de4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:05.965544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754b820-d05e-493c-a5d9-462e957f1de4 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:05.968913 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754b820-d05e-493c-a5d9-462e957f1de4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:05.968913 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754b820-d05e-493c-a5d9-462e957f1de4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:05.968913 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754b820-d05e-493c-a5d9-462e957f1de4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:05.984267 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 17/33] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:05.985568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754b820-d05e-493c-a5d9-462e957f1de4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:06.012802 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34a76e-d820-4b9c-b227-16d2484743b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:06.013428 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34a76e-d820-4b9c-b227-16d2484743b8 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:06.013709 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34a76e-d820-4b9c-b227-16d2484743b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:06.013979 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34a76e-d820-4b9c-b227-16d2484743b8 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:06.014702 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34a76e-d820-4b9c-b227-16d2484743b8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:06.016125 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34a76e-d820-4b9c-b227-16d2484743b8 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:06.041059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:06.143164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 17/34] 10.222.0.33 () {60 vars in 1131 bytes} [Tue Oct 4 12:34:05 2022] POST /identity/v3/roles => generated 225 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:34:06.145290 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dd39a-98d3-415e-8d16-89a3b40d1c3c None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:06.145936 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dd39a-98d3-415e-8d16-89a3b40d1c3c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:06.145936 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674dd39a-98d3-415e-8d16-89a3b40d1c3c None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:06.147542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 18/35] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:06.148167 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674dd39a-98d3-415e-8d16-89a3b40d1c3c None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:06.165980 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754b820-d05e-493c-a5d9-462e957f1de4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:06.217008 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:06.247325 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbe647-5f7f-45d6-bac1-e003c9386336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:06.248273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbe647-5f7f-45d6-bac1-e003c9386336 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:06.248690 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbe647-5f7f-45d6-bac1-e003c9386336 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:06.249148 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbe647-5f7f-45d6-bac1-e003c9386336 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:06.253914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbe647-5f7f-45d6-bac1-e003c9386336 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:06.253914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbe647-5f7f-45d6-bac1-e003c9386336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:06.253914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbe647-5f7f-45d6-bac1-e003c9386336 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:06.276966 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 18/36] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:06.286427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b18a8c-1cdb-404e-8d0a-b61928edb354 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:06.286427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b18a8c-1cdb-404e-8d0a-b61928edb354 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:06.286427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b18a8c-1cdb-404e-8d0a-b61928edb354 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:06.286427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 19/37] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:06.286427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fffab1-76bc-4606-894a-ce9637b2bc27 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:06.288398 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fffab1-76bc-4606-894a-ce9637b2bc27 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:06.290814 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fffab1-76bc-4606-894a-ce9637b2bc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:06.323279 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-37fffab1-76bc-4606-894a-ce9637b2bc27 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:06.326615 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-37fffab1-76bc-4606-894a-ce9637b2bc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:06.330282 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674dd39a-98d3-415e-8d16-89a3b40d1c3c None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:06.393397 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ddd10-ac7c-4bb5-9a70-60668baa2fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:06.394267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ddd10-ac7c-4bb5-9a70-60668baa2fcf None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:06.394916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ddd10-ac7c-4bb5-9a70-60668baa2fcf None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:06.395379 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ddd10-ac7c-4bb5-9a70-60668baa2fcf None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:06.396770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ddd10-ac7c-4bb5-9a70-60668baa2fcf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:06.399430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ddd10-ac7c-4bb5-9a70-60668baa2fcf None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:06.442729 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 19/38] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:34:06 2022] POST /identity/v3/projects => generated 301 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:34:06.444657 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ddd10-ac7c-4bb5-9a70-60668baa2fcf None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:06.459098 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 20/39] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:06.461263 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86ff94-93fe-4b9c-a6a8-5fc7928842d5 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:06.461701 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86ff94-93fe-4b9c-a6a8-5fc7928842d5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:06.462504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86ff94-93fe-4b9c-a6a8-5fc7928842d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:06.508017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4c86ff94-93fe-4b9c-a6a8-5fc7928842d5 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:06.510296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4c86ff94-93fe-4b9c-a6a8-5fc7928842d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:06.544621 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:06.652649 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 21/40] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:06.820435 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:06.827555 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ddd10-ac7c-4bb5-9a70-60668baa2fcf None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:06.892913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93540415-d9a7-41f0-b542-6ed63aaddb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:06.893961 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93540415-d9a7-41f0-b542-6ed63aaddb07 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:06.894739 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93540415-d9a7-41f0-b542-6ed63aaddb07 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:06.895231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93540415-d9a7-41f0-b542-6ed63aaddb07 None admin] PATH_INFO: `/v3/users/ba3be3b6260847d081e4ed3e9d15d818` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:06.897426 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93540415-d9a7-41f0-b542-6ed63aaddb07 None admin] RBAC: Authorizing `identity:get_user(user_id=ba3be3b6260847d081e4ed3e9d15d818)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:06.900051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93540415-d9a7-41f0-b542-6ed63aaddb07 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:06.903128 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 20/41] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:06 2022] GET /identity/v3/users/ba3be3b6260847d081e4ed3e9d15d818 => generated 247 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:06.911685 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86ff94-93fe-4b9c-a6a8-5fc7928842d5 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:06.932267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93540415-d9a7-41f0-b542-6ed63aaddb07 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:07.050511 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86ff94-93fe-4b9c-a6a8-5fc7928842d5 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:07.085153 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93540415-d9a7-41f0-b542-6ed63aaddb07 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:07.087177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364bdc6-7be4-4f22-998e-51f1f293bada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:07.087830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364bdc6-7be4-4f22-998e-51f1f293bada None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:07.088110 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364bdc6-7be4-4f22-998e-51f1f293bada None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:07.088387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364bdc6-7be4-4f22-998e-51f1f293bada None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:07.089870 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364bdc6-7be4-4f22-998e-51f1f293bada None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:07.091416 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364bdc6-7be4-4f22-998e-51f1f293bada None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:07.093058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 22/42] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:07.093836 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364bdc6-7be4-4f22-998e-51f1f293bada None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:07.116069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:07.146536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48145265-577d-4645-bdf9-28fb20b274d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:07.147586 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48145265-577d-4645-bdf9-28fb20b274d2 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:07.148060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48145265-577d-4645-bdf9-28fb20b274d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:07.148550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48145265-577d-4645-bdf9-28fb20b274d2 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:07.149778 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48145265-577d-4645-bdf9-28fb20b274d2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:07.152494 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48145265-577d-4645-bdf9-28fb20b274d2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:07.178234 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 21/43] 10.222.0.33 () {60 vars in 1131 bytes} [Tue Oct 4 12:34:06 2022] POST /identity/v3/roles => generated 223 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:34:07.178234 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48145265-577d-4645-bdf9-28fb20b274d2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:07.220914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364bdc6-7be4-4f22-998e-51f1f293bada None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:07.226210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:07.288693 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeac483-a1b9-42fb-a9ee-f0a4a2997c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:07.289719 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeac483-a1b9-42fb-a9ee-f0a4a2997c16 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:07.290170 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeac483-a1b9-42fb-a9ee-f0a4a2997c16 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:07.291823 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeac483-a1b9-42fb-a9ee-f0a4a2997c16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:07.292813 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeac483-a1b9-42fb-a9ee-f0a4a2997c16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:07.295799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeac483-a1b9-42fb-a9ee-f0a4a2997c16 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:07.302198 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 23/44] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:07.308349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeac483-a1b9-42fb-a9ee-f0a4a2997c16 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:07.344305 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48145265-577d-4645-bdf9-28fb20b274d2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:07.394069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d48b8c-0c70-48c6-ba25-8607d762441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:07.395674 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d48b8c-0c70-48c6-ba25-8607d762441a None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:07.396083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d48b8c-0c70-48c6-ba25-8607d762441a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:07.396491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d48b8c-0c70-48c6-ba25-8607d762441a None admin] PATH_INFO: `/v3/domains/default/users/ba3be3b6260847d081e4ed3e9d15d818/roles/fc68d072459d4720b9666c93bd535d63` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:07.399556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d48b8c-0c70-48c6-ba25-8607d762441a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ba3be3b6260847d081e4ed3e9d15d818, role_id=fc68d072459d4720b9666c93bd535d63)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:07.402132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d48b8c-0c70-48c6-ba25-8607d762441a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:07.428098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 22/45] 10.222.0.33 () {58 vars in 1361 bytes} [Tue Oct 4 12:34:07 2022] PUT /identity/v3/domains/default/users/ba3be3b6260847d081e4ed3e9d15d818/roles/fc68d072459d4720b9666c93bd535d63 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:34:07.488520 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:07.510235 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeac483-a1b9-42fb-a9ee-f0a4a2997c16 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:07.553440 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6de54-3664-489e-95ef-de5d2390cca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:07.554249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6de54-3664-489e-95ef-de5d2390cca0 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:07.554631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6de54-3664-489e-95ef-de5d2390cca0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:07.554992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6de54-3664-489e-95ef-de5d2390cca0 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:07.555807 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6de54-3664-489e-95ef-de5d2390cca0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:07.557486 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6de54-3664-489e-95ef-de5d2390cca0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:07.614168 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 24/46] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:34:07 2022] POST /identity/v3/projects => generated 312 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:34:09.924289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8832c-1451-4ac7-a439-12862017e09a None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:09.924827 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8832c-1451-4ac7-a439-12862017e09a None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:09.924827 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8832c-1451-4ac7-a439-12862017e09a None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:09.926268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 23/47] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:34:09.933259 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d9302-5f4a-49b5-bf54-969e8111ea7f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:09.934390 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d9302-5f4a-49b5-bf54-969e8111ea7f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:09.934708 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d9302-5f4a-49b5-bf54-969e8111ea7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:09.965339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-172d9302-5f4a-49b5-bf54-969e8111ea7f None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:09.967701 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-172d9302-5f4a-49b5-bf54-969e8111ea7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:10.078098 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 25/48] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:10.106284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edd7b7-0751-481e-ba72-e8d16499568a None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:10.106750 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edd7b7-0751-481e-ba72-e8d16499568a None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:10.107215 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56edd7b7-0751-481e-ba72-e8d16499568a None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:10.109593 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 24/49] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:10.117666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607124b0-d6d0-4df1-a348-9f6c283d8bf3 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:10.118134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607124b0-d6d0-4df1-a348-9f6c283d8bf3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:10.118500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607124b0-d6d0-4df1-a348-9f6c283d8bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:10.140018 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-607124b0-d6d0-4df1-a348-9f6c283d8bf3 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:10.141471 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-607124b0-d6d0-4df1-a348-9f6c283d8bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:10.215071 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:10.226328 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 26/50] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:10.362650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edd7b7-0751-481e-ba72-e8d16499568a None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:10.460292 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:10.479763 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56edd7b7-0751-481e-ba72-e8d16499568a None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:10.516621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cddb4-ae32-4c88-8236-26abf1327bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:10.517261 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cddb4-ae32-4c88-8236-26abf1327bf2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:10.517542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cddb4-ae32-4c88-8236-26abf1327bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:10.517812 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cddb4-ae32-4c88-8236-26abf1327bf2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:10.529701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82cddb4-ae32-4c88-8236-26abf1327bf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-b82cddb4-ae32-4c88-8236-26abf1327bf2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:10.535051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:10.535954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:10.536938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 25/51] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:34:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:10.542159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607124b0-d6d0-4df1-a348-9f6c283d8bf3 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:10.731618 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cddb4-ae32-4c88-8236-26abf1327bf2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:10.770792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607124b0-d6d0-4df1-a348-9f6c283d8bf3 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:10.822276 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f0579-00ac-403d-a6d0-2b8e7c340bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:10.822917 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f0579-00ac-403d-a6d0-2b8e7c340bcf None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:10.823186 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f0579-00ac-403d-a6d0-2b8e7c340bcf None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:10.823463 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f0579-00ac-403d-a6d0-2b8e7c340bcf None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:10.824676 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f0579-00ac-403d-a6d0-2b8e7c340bcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:10.824981 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f0579-00ac-403d-a6d0-2b8e7c340bcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:10.826439 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f0579-00ac-403d-a6d0-2b8e7c340bcf None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:10.835918 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 27/52] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:10.843094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f0579-00ac-403d-a6d0-2b8e7c340bcf None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:10.862880 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cddb4-ae32-4c88-8236-26abf1327bf2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:10.907429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:10.921648 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdbbf2-6776-4b14-91a0-cf0517614ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:10.922481 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdbbf2-6776-4b14-91a0-cf0517614ce7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:10.922849 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdbbf2-6776-4b14-91a0-cf0517614ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:10.923211 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdbbf2-6776-4b14-91a0-cf0517614ce7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:10.924537 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdbbf2-6776-4b14-91a0-cf0517614ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:10.926621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdbbf2-6776-4b14-91a0-cf0517614ce7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:10.928578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 26/53] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:10.934868 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdbbf2-6776-4b14-91a0-cf0517614ce7 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:10.954004 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:10.955543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f0579-00ac-403d-a6d0-2b8e7c340bcf None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:10.997782 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81456b-9423-447f-90c3-d3dd165d3581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:10.998763 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81456b-9423-447f-90c3-d3dd165d3581 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:10.999227 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81456b-9423-447f-90c3-d3dd165d3581 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:10.999578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81456b-9423-447f-90c3-d3dd165d3581 None admin] PATH_INFO: `/v3/users/ba3be3b6260847d081e4ed3e9d15d818` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:11.001918 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81456b-9423-447f-90c3-d3dd165d3581 None admin] RBAC: Authorizing `identity:get_user(user_id=ba3be3b6260847d081e4ed3e9d15d818)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:11.004540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81456b-9423-447f-90c3-d3dd165d3581 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:11.010336 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 28/54] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:10 2022] GET /identity/v3/users/ba3be3b6260847d081e4ed3e9d15d818 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:11.013207 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81456b-9423-447f-90c3-d3dd165d3581 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:11.076368 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdbbf2-6776-4b14-91a0-cf0517614ce7 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:11.118745 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031c8df-96db-4ad5-8864-f4337797b480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:11.119476 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031c8df-96db-4ad5-8864-f4337797b480 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:11.119836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031c8df-96db-4ad5-8864-f4337797b480 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:11.120229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031c8df-96db-4ad5-8864-f4337797b480 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:11.120980 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031c8df-96db-4ad5-8864-f4337797b480 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:11.123210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031c8df-96db-4ad5-8864-f4337797b480 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:11.138341 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81456b-9423-447f-90c3-d3dd165d3581 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:11.158324 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 27/55] 10.222.0.33 () {60 vars in 1140 bytes} [Tue Oct 4 12:34:10 2022] POST /identity/v3/projects => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:34:11.174300 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:11.185004 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a7529-de25-4ceb-8d40-a951814b6067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:11.186123 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a7529-de25-4ceb-8d40-a951814b6067 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:11.186570 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a7529-de25-4ceb-8d40-a951814b6067 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:11.186923 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a7529-de25-4ceb-8d40-a951814b6067 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:11.188354 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a7529-de25-4ceb-8d40-a951814b6067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:11.189952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a7529-de25-4ceb-8d40-a951814b6067 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:11.191842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 29/56] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:11.198253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031c8df-96db-4ad5-8864-f4337797b480 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:11.300556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:11.348258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031c8df-96db-4ad5-8864-f4337797b480 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:11.391444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4ee58-f387-41d5-b720-03780763b8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:11.392249 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4ee58-f387-41d5-b720-03780763b8ee None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:11.392601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4ee58-f387-41d5-b720-03780763b8ee None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:11.392971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4ee58-f387-41d5-b720-03780763b8ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:11.393841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4ee58-f387-41d5-b720-03780763b8ee None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.domain.id=default, user.id=ba3be3b6260847d081e4ed3e9d15d818) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:11.394328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4ee58-f387-41d5-b720-03780763b8ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:11.396238 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4ee58-f387-41d5-b720-03780763b8ee None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:11.404887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 28/57] 10.222.0.33 () {56 vars in 1322 bytes} [Tue Oct 4 12:34:11 2022] GET /identity/v3/role_assignments?user.id=ba3be3b6260847d081e4ed3e9d15d818&scope.domain.id=default&role.id=fc68d072459d4720b9666c93bd535d63 => generated 530 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:13.463566 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63be73-d1b5-4ddc-bf11-8de08c5e3289 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:13.464135 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63be73-d1b5-4ddc-bf11-8de08c5e3289 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:13.464135 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63be73-d1b5-4ddc-bf11-8de08c5e3289 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:13.465995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 30/58] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:34:13.471659 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a5b12-2130-4084-85ec-90bd7e6ebdfa None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:13.472174 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a5b12-2130-4084-85ec-90bd7e6ebdfa None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:13.472297 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a5b12-2130-4084-85ec-90bd7e6ebdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:13.491988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-9a5a5b12-2130-4084-85ec-90bd7e6ebdfa None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:13.493573 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-9a5a5b12-2130-4084-85ec-90bd7e6ebdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:13.568648 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 29/59] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:34:13.842254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63be73-d1b5-4ddc-bf11-8de08c5e3289 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:13.955427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63be73-d1b5-4ddc-bf11-8de08c5e3289 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:13.991694 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c089d-7913-45da-8c6b-1c49b520ad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:13.992335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c089d-7913-45da-8c6b-1c49b520ad86 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:13.992615 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c089d-7913-45da-8c6b-1c49b520ad86 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:13.992890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c089d-7913-45da-8c6b-1c49b520ad86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:13.994032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c089d-7913-45da-8c6b-1c49b520ad86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:13.995526 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c089d-7913-45da-8c6b-1c49b520ad86 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:13.997144 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 31/60] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:14.002974 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a5b12-2130-4084-85ec-90bd7e6ebdfa None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:14.065416 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:14.110846 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a5b12-2130-4084-85ec-90bd7e6ebdfa None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:14.148269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779a357-e512-41ac-a64c-6fb75dd6ed5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:14.148923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779a357-e512-41ac-a64c-6fb75dd6ed5b None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:14.149235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779a357-e512-41ac-a64c-6fb75dd6ed5b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:14.149542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779a357-e512-41ac-a64c-6fb75dd6ed5b None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:14.150354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779a357-e512-41ac-a64c-6fb75dd6ed5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:14.151885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779a357-e512-41ac-a64c-6fb75dd6ed5b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:14.181496 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 30/61] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:34:14 2022] POST /identity/v3/users => generated 275 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:34:18.498088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646918d-20cc-42a7-99c5-134bec222208 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:18.498603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646918d-20cc-42a7-99c5-134bec222208 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:18.498675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646918d-20cc-42a7-99c5-134bec222208 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:18.500013 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 32/62] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:34:18.516459 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451e77-4c60-47a3-aa35-3580001f27ff None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:18.516941 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451e77-4c60-47a3-aa35-3580001f27ff None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:18.516941 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451e77-4c60-47a3-aa35-3580001f27ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:18.542613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-9b451e77-4c60-47a3-aa35-3580001f27ff None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:18.544400 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-9b451e77-4c60-47a3-aa35-3580001f27ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:18.649642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 31/63] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:18.769506 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6141fe-d641-4de3-a521-2f05be2a237e None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:18.770397 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6141fe-d641-4de3-a521-2f05be2a237e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:18.770879 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6141fe-d641-4de3-a521-2f05be2a237e None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:18.772777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 33/64] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:18.778819 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2e158-7b35-440a-a4f4-677468f6197e None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:18.779432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2e158-7b35-440a-a4f4-677468f6197e None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:18.779906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2e158-7b35-440a-a4f4-677468f6197e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:18.807213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-d6c2e158-7b35-440a-a4f4-677468f6197e None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:18.814311 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-d6c2e158-7b35-440a-a4f4-677468f6197e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:18.871519 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:18.907462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 32/65] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:19.219994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6141fe-d641-4de3-a521-2f05be2a237e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:19.234506 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2e158-7b35-440a-a4f4-677468f6197e None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:19.333845 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:19.387795 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2e158-7b35-440a-a4f4-677468f6197e None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:19.395818 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6141fe-d641-4de3-a521-2f05be2a237e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:19.428967 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec816308-a12f-41bf-bd23-bf5fb5e369cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:19.430023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec816308-a12f-41bf-bd23-bf5fb5e369cd None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:19.430541 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec816308-a12f-41bf-bd23-bf5fb5e369cd None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:19.431040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec816308-a12f-41bf-bd23-bf5fb5e369cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:19.444675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec816308-a12f-41bf-bd23-bf5fb5e369cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:19.447010 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb93d80-dcf6-4c6a-bed0-4077e49a0937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:19.447683 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb93d80-dcf6-4c6a-bed0-4077e49a0937 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:19.447968 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb93d80-dcf6-4c6a-bed0-4077e49a0937 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:19.448247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb93d80-dcf6-4c6a-bed0-4077e49a0937 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-ec816308-a12f-41bf-bd23-bf5fb5e369cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:19.451797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:19.452692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:19.453451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:19.453451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:19.453451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:19.453451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:19.453451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:19.453451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:19.453451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:19.453451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:19.453451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:19.453451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:19.453451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:19.453451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:19.455914 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 33/66] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:34:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:19.458161 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48eca-2314-497d-bfbd-37b3b817124d None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:19.458726 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48eca-2314-497d-bfbd-37b3b817124d None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:19.459204 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48eca-2314-497d-bfbd-37b3b817124d None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:19.460864 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 34/67] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:19.462131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d48eca-2314-497d-bfbd-37b3b817124d None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:19.478408 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb93d80-dcf6-4c6a-bed0-4077e49a0937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-5fb93d80-dcf6-4c6a-bed0-4077e49a0937 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:19.487636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:19.488558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:19.489313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 34/68] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:19.490198 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ea310-8fa8-457c-8ff8-ce6891c7c874 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:19.490636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ea310-8fa8-457c-8ff8-ce6891c7c874 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:19.491080 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ea310-8fa8-457c-8ff8-ce6891c7c874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:19.517893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:19.520642 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-da0ea310-8fa8-457c-8ff8-ce6891c7c874 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:19.522612 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-da0ea310-8fa8-457c-8ff8-ce6891c7c874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:19.611769 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d48eca-2314-497d-bfbd-37b3b817124d None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:19.629101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 35/69] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:19.630477 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ea310-8fa8-457c-8ff8-ce6891c7c874 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:19.660248 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723463e-a087-4417-af7b-a6897d09946a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:19.661236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723463e-a087-4417-af7b-a6897d09946a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:19.661670 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723463e-a087-4417-af7b-a6897d09946a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:19.662256 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723463e-a087-4417-af7b-a6897d09946a None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:19.662978 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723463e-a087-4417-af7b-a6897d09946a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:19.663439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723463e-a087-4417-af7b-a6897d09946a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:19.665265 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723463e-a087-4417-af7b-a6897d09946a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:19.680588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 35/70] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:19.682708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d135cf-5a40-44ed-a973-c04dbc151635 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:19.682995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d135cf-5a40-44ed-a973-c04dbc151635 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:19.683278 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d135cf-5a40-44ed-a973-c04dbc151635 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:19.685218 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 36/71] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:19.691741 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d135cf-5a40-44ed-a973-c04dbc151635 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:19.742421 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:19.821045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:19.826673 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d135cf-5a40-44ed-a973-c04dbc151635 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:19.827077 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ea310-8fa8-457c-8ff8-ce6891c7c874 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:19.864766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03461fdd-864c-4fa8-9f6c-ba6e01f1dca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:19.865580 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03461fdd-864c-4fa8-9f6c-ba6e01f1dca6 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:19.865829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03461fdd-864c-4fa8-9f6c-ba6e01f1dca6 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:19.866133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03461fdd-864c-4fa8-9f6c-ba6e01f1dca6 None admin] PATH_INFO: `/v3/users/ba3be3b6260847d081e4ed3e9d15d818` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:19.867446 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03461fdd-864c-4fa8-9f6c-ba6e01f1dca6 None admin] RBAC: Authorizing `identity:get_user(user_id=ba3be3b6260847d081e4ed3e9d15d818)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:19.869142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03461fdd-864c-4fa8-9f6c-ba6e01f1dca6 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:19.870841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 37/72] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:19 2022] GET /identity/v3/users/ba3be3b6260847d081e4ed3e9d15d818 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:19.872803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e336b3e-7831-4cf6-a08a-e369deaa5b6d None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:19.873085 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e336b3e-7831-4cf6-a08a-e369deaa5b6d None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:19.873358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e336b3e-7831-4cf6-a08a-e369deaa5b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:19.892661 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-0e336b3e-7831-4cf6-a08a-e369deaa5b6d None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:19.893891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-0e336b3e-7831-4cf6-a08a-e369deaa5b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:19.896360 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4519d1-f72b-4de2-a626-2e06103282a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:19.897416 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4519d1-f72b-4de2-a626-2e06103282a6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:19.897919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4519d1-f72b-4de2-a626-2e06103282a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:19.898460 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4519d1-f72b-4de2-a626-2e06103282a6 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:19.899711 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4519d1-f72b-4de2-a626-2e06103282a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:19.900276 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4519d1-f72b-4de2-a626-2e06103282a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:19.902298 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4519d1-f72b-4de2-a626-2e06103282a6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:19.920430 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 36/73] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:19.920430 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83085bf7-1620-49a6-ac4b-e632c2ccc289 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:19.920430 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83085bf7-1620-49a6-ac4b-e632c2ccc289 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:19.920430 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83085bf7-1620-49a6-ac4b-e632c2ccc289 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:19.920430 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 37/74] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:19.920430 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83085bf7-1620-49a6-ac4b-e632c2ccc289 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:19.990001 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 38/75] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:19.991051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e336b3e-7831-4cf6-a08a-e369deaa5b6d None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:20.094953 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83085bf7-1620-49a6-ac4b-e632c2ccc289 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:20.101372 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:20.109337 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e336b3e-7831-4cf6-a08a-e369deaa5b6d None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:20.118367 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:20.143642 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107981f2-992f-4a09-863f-5b761e722f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:20.144515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107981f2-992f-4a09-863f-5b761e722f8f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:20.145062 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107981f2-992f-4a09-863f-5b761e722f8f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:20.145334 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107981f2-992f-4a09-863f-5b761e722f8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:20.171258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa375fe0-2dfa-425b-8005-09aaa16df80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:20.171890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa375fe0-2dfa-425b-8005-09aaa16df80e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:20.172177 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa375fe0-2dfa-425b-8005-09aaa16df80e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:20.172465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa375fe0-2dfa-425b-8005-09aaa16df80e None admin] PATH_INFO: `/v3/users/7563b6e0ea29491291eb70a0be54b165` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:20.197732 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa375fe0-2dfa-425b-8005-09aaa16df80e None admin] RBAC: Authorizing `identity:get_user(user_id=7563b6e0ea29491291eb70a0be54b165)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:20.201145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa375fe0-2dfa-425b-8005-09aaa16df80e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:20.208416 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 39/76] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:19 2022] GET /identity/v3/users/7563b6e0ea29491291eb70a0be54b165 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:20.208416 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58195f-fe5d-48d5-b007-06c75a240689 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:20.208416 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58195f-fe5d-48d5-b007-06c75a240689 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:20.208416 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58195f-fe5d-48d5-b007-06c75a240689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:20.229963 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-fa58195f-fe5d-48d5-b007-06c75a240689 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:20.231263 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-fa58195f-fe5d-48d5-b007-06c75a240689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:20.232924 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107981f2-992f-4a09-863f-5b761e722f8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:20.235584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107981f2-992f-4a09-863f-5b761e722f8f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:20.315206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 40/77] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:20.315206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58195f-fe5d-48d5-b007-06c75a240689 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:20.406119 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 38/78] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:20.406119 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107981f2-992f-4a09-863f-5b761e722f8f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:20.418609 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:20.423792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:20.440025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58195f-fe5d-48d5-b007-06c75a240689 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:20.483052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa369a-916d-48b1-9783-7c44ae0d8ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:20.483709 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa369a-916d-48b1-9783-7c44ae0d8ebb None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:20.483991 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa369a-916d-48b1-9783-7c44ae0d8ebb None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:20.484265 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa369a-916d-48b1-9783-7c44ae0d8ebb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:20.501757 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa369a-916d-48b1-9783-7c44ae0d8ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-5ffa369a-916d-48b1-9783-7c44ae0d8ebb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:20.507293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:20.509059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 41/79] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:34:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:20.510613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa369a-916d-48b1-9783-7c44ae0d8ebb None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:20.537819 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107981f2-992f-4a09-863f-5b761e722f8f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:20.581609 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d013fdd-4ddc-4b3f-b9a7-bb0013947cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:20.582408 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d013fdd-4ddc-4b3f-b9a7-bb0013947cde None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:20.582700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d013fdd-4ddc-4b3f-b9a7-bb0013947cde None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:20.582982 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d013fdd-4ddc-4b3f-b9a7-bb0013947cde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:20.663507 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa369a-916d-48b1-9783-7c44ae0d8ebb None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:20.672689 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d013fdd-4ddc-4b3f-b9a7-bb0013947cde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:20.675300 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d013fdd-4ddc-4b3f-b9a7-bb0013947cde None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:20.694720 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:20.701585 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76012f-ad39-42a9-943d-1899e5499a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:20.702256 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76012f-ad39-42a9-943d-1899e5499a3f None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:20.702537 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76012f-ad39-42a9-943d-1899e5499a3f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:20.702812 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76012f-ad39-42a9-943d-1899e5499a3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:20.703993 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76012f-ad39-42a9-943d-1899e5499a3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:20.705477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76012f-ad39-42a9-943d-1899e5499a3f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:20.707127 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 42/80] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:20.707929 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76012f-ad39-42a9-943d-1899e5499a3f None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:20.724760 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:20.798864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 39/81] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:20.799716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d013fdd-4ddc-4b3f-b9a7-bb0013947cde None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:20.834460 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76012f-ad39-42a9-943d-1899e5499a3f None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:20.885569 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5437a-b90f-4798-a48e-3f5b27f7a59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:20.886237 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5437a-b90f-4798-a48e-3f5b27f7a59b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:20.886519 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5437a-b90f-4798-a48e-3f5b27f7a59b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:20.887372 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5437a-b90f-4798-a48e-3f5b27f7a59b None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:20.888576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5437a-b90f-4798-a48e-3f5b27f7a59b None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:20.890063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5437a-b90f-4798-a48e-3f5b27f7a59b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:20.891721 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 43/82] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:20 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:20.894024 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5437a-b90f-4798-a48e-3f5b27f7a59b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:20.909183 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d013fdd-4ddc-4b3f-b9a7-bb0013947cde None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:20.925238 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:20.952223 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc29ed-8e7f-4c13-ad57-a64cf1b15636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:20.952839 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc29ed-8e7f-4c13-ad57-a64cf1b15636 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:20.953116 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc29ed-8e7f-4c13-ad57-a64cf1b15636 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:20.953387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc29ed-8e7f-4c13-ad57-a64cf1b15636 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:20.954087 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc29ed-8e7f-4c13-ad57-a64cf1b15636 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:20.954396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc29ed-8e7f-4c13-ad57-a64cf1b15636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:20.955786 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc29ed-8e7f-4c13-ad57-a64cf1b15636 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:20.965153 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 40/83] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:34:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:20.967259 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc29ed-8e7f-4c13-ad57-a64cf1b15636 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:20.987628 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:21.014555 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5437a-b90f-4798-a48e-3f5b27f7a59b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:21.052107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f2564-ab8a-4765-92db-53cfacf2bfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:21.052760 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f2564-ab8a-4765-92db-53cfacf2bfa1 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:21.053041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f2564-ab8a-4765-92db-53cfacf2bfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:21.053312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f2564-ab8a-4765-92db-53cfacf2bfa1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:21.065701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f2564-ab8a-4765-92db-53cfacf2bfa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-626f2564-ab8a-4765-92db-53cfacf2bfa1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:21.071543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:21.072465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:21.073210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 44/84] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:21.073838 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f2564-ab8a-4765-92db-53cfacf2bfa1 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:21.107460 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc29ed-8e7f-4c13-ad57-a64cf1b15636 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:21.164594 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff15711-129b-4db2-b664-7ae920550459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:21.165641 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff15711-129b-4db2-b664-7ae920550459 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:21.166140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff15711-129b-4db2-b664-7ae920550459 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:21.166626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff15711-129b-4db2-b664-7ae920550459 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:21.168196 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff15711-129b-4db2-b664-7ae920550459 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:21.171121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff15711-129b-4db2-b664-7ae920550459 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:21.201364 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f2564-ab8a-4765-92db-53cfacf2bfa1 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:21.217397 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 41/85] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:34:20 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:34:21.218916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff15711-129b-4db2-b664-7ae920550459 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:21.279498 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667717b8-eb71-466e-bfc1-577dfe60e4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:21.280476 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667717b8-eb71-466e-bfc1-577dfe60e4c2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:21.280800 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667717b8-eb71-466e-bfc1-577dfe60e4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:21.281216 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667717b8-eb71-466e-bfc1-577dfe60e4c2 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:21.283347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667717b8-eb71-466e-bfc1-577dfe60e4c2 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:21.283882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:21.286547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667717b8-eb71-466e-bfc1-577dfe60e4c2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:21.296451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 45/86] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:21 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:21.296451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667717b8-eb71-466e-bfc1-577dfe60e4c2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:21.317542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:21.419045 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff15711-129b-4db2-b664-7ae920550459 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:21.452260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667717b8-eb71-466e-bfc1-577dfe60e4c2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:21.487158 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f6cf5-f918-41ad-854f-12d600d2794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:21.488089 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f6cf5-f918-41ad-854f-12d600d2794d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:21.488504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f6cf5-f918-41ad-854f-12d600d2794d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:21.488929 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f6cf5-f918-41ad-854f-12d600d2794d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:21.490187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f6cf5-f918-41ad-854f-12d600d2794d None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=3f0bb97051994c89b3544850957b48be, user.id=ba3be3b6260847d081e4ed3e9d15d818) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:21.490669 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f6cf5-f918-41ad-854f-12d600d2794d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:21.493192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f6cf5-f918-41ad-854f-12d600d2794d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:21.508521 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 42/87] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:21 2022] GET /identity/v3/role_assignments?user.id=ba3be3b6260847d081e4ed3e9d15d818&scope.project.id=3f0bb97051994c89b3544850957b48be&role.id=fc68d072459d4720b9666c93bd535d63 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:21.508706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11f624-aa93-4274-9b95-9cad166d139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:21.509563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11f624-aa93-4274-9b95-9cad166d139c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:21.509684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f6cf5-f918-41ad-854f-12d600d2794d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:21.509995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11f624-aa93-4274-9b95-9cad166d139c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:21.514935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11f624-aa93-4274-9b95-9cad166d139c None admin] PATH_INFO: `/v3/users/7563b6e0ea29491291eb70a0be54b165` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:21.523487 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11f624-aa93-4274-9b95-9cad166d139c None admin] RBAC: Authorizing `identity:get_user(user_id=7563b6e0ea29491291eb70a0be54b165)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:21.526433 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11f624-aa93-4274-9b95-9cad166d139c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:21.530970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 46/88] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:21 2022] GET /identity/v3/users/7563b6e0ea29491291eb70a0be54b165 => generated 275 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:21.532532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11f624-aa93-4274-9b95-9cad166d139c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:21.698014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:21.713554 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f6cf5-f918-41ad-854f-12d600d2794d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:21.727589 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:21.760986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11f624-aa93-4274-9b95-9cad166d139c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:21.808212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef42720-eb6c-4fc8-aa2a-2fc887a88abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:21.809299 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef42720-eb6c-4fc8-aa2a-2fc887a88abe None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:21.809791 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef42720-eb6c-4fc8-aa2a-2fc887a88abe None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:21.810337 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef42720-eb6c-4fc8-aa2a-2fc887a88abe None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:21.811580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef42720-eb6c-4fc8-aa2a-2fc887a88abe None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:21.812263 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef42720-eb6c-4fc8-aa2a-2fc887a88abe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:21.814879 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef42720-eb6c-4fc8-aa2a-2fc887a88abe None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:21.837404 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 43/89] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:21.838997 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef42720-eb6c-4fc8-aa2a-2fc887a88abe None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:21.842638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ed5db-26bd-4fe6-8d10-da78bdcd0e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:21.843660 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ed5db-26bd-4fe6-8d10-da78bdcd0e21 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:21.844278 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ed5db-26bd-4fe6-8d10-da78bdcd0e21 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:21.845076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ed5db-26bd-4fe6-8d10-da78bdcd0e21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:21.846538 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ed5db-26bd-4fe6-8d10-da78bdcd0e21 None admin] RBAC: Adding query filter params (role.id=d18e4897bf3648498a472a2ea8174f8b, scope.project.id=3f0bb97051994c89b3544850957b48be, user.id=7563b6e0ea29491291eb70a0be54b165) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:21.847091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ed5db-26bd-4fe6-8d10-da78bdcd0e21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:21.849931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ed5db-26bd-4fe6-8d10-da78bdcd0e21 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:21.863630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 47/90] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:21 2022] GET /identity/v3/role_assignments?user.id=7563b6e0ea29491291eb70a0be54b165&scope.project.id=3f0bb97051994c89b3544850957b48be&role.id=d18e4897bf3648498a472a2ea8174f8b => generated 259 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:21.865218 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ed5db-26bd-4fe6-8d10-da78bdcd0e21 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:22.080884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef42720-eb6c-4fc8-aa2a-2fc887a88abe None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:22.136847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ed5db-26bd-4fe6-8d10-da78bdcd0e21 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:22.152060 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a1dce-bcf4-487b-b2c0-d703f613d5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:22.153205 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a1dce-bcf4-487b-b2c0-d703f613d5e1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:22.153780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a1dce-bcf4-487b-b2c0-d703f613d5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:22.154368 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a1dce-bcf4-487b-b2c0-d703f613d5e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:22.221612 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:22.228159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5519ad-99e6-4309-a6ba-5cbb6abbeac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:22.229221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5519ad-99e6-4309-a6ba-5cbb6abbeac0 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:22.229631 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5519ad-99e6-4309-a6ba-5cbb6abbeac0 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:22.230069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5519ad-99e6-4309-a6ba-5cbb6abbeac0 None admin] PATH_INFO: `/v3/users/7563b6e0ea29491291eb70a0be54b165` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:22.235494 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5519ad-99e6-4309-a6ba-5cbb6abbeac0 None admin] RBAC: Authorizing `identity:get_user(user_id=7563b6e0ea29491291eb70a0be54b165)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:22.238147 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5519ad-99e6-4309-a6ba-5cbb6abbeac0 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:22.241260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 48/91] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:21 2022] GET /identity/v3/users/7563b6e0ea29491291eb70a0be54b165 => generated 275 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:22.248320 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5519ad-99e6-4309-a6ba-5cbb6abbeac0 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:22.286537 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:22.323005 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a1dce-bcf4-487b-b2c0-d703f613d5e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:22.326605 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a1dce-bcf4-487b-b2c0-d703f613d5e1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:22.541606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5519ad-99e6-4309-a6ba-5cbb6abbeac0 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:22.592124 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 44/92] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 754 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:22.601890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a1dce-bcf4-487b-b2c0-d703f613d5e1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:22.656344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb319a-9469-4c30-b4e7-a2b1ddd3b6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:22.657457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb319a-9469-4c30-b4e7-a2b1ddd3b6a6 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:22.657947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb319a-9469-4c30-b4e7-a2b1ddd3b6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:22.658895 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb319a-9469-4c30-b4e7-a2b1ddd3b6a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:22.721556 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:22.847569 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a1dce-bcf4-487b-b2c0-d703f613d5e1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:22.911312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb319a-9469-4c30-b4e7-a2b1ddd3b6a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:22.917310 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb319a-9469-4c30-b4e7-a2b1ddd3b6a6 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:22.947806 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:22.991817 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2787b-c78c-4e27-86b7-e7de1054991c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:22.995396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2787b-c78c-4e27-86b7-e7de1054991c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:22.996560 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2787b-c78c-4e27-86b7-e7de1054991c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:22.999381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc2787b-c78c-4e27-86b7-e7de1054991c None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:23.002350 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2787b-c78c-4e27-86b7-e7de1054991c None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:23.007010 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc2787b-c78c-4e27-86b7-e7de1054991c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:23.012213 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 45/93] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:22 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:23.033074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2787b-c78c-4e27-86b7-e7de1054991c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:23.235898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 49/94] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 989 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 12:34:23.244180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb319a-9469-4c30-b4e7-a2b1ddd3b6a6 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:23.432768 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc2787b-c78c-4e27-86b7-e7de1054991c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:23.545735 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:23.588996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb319a-9469-4c30-b4e7-a2b1ddd3b6a6 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:23.592314 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42bd5a-1db6-4512-a437-7ca71bc6ce52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:23.593260 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42bd5a-1db6-4512-a437-7ca71bc6ce52 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:23.593732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42bd5a-1db6-4512-a437-7ca71bc6ce52 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:23.594239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42bd5a-1db6-4512-a437-7ca71bc6ce52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:23.603390 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42bd5a-1db6-4512-a437-7ca71bc6ce52 None admin] RBAC: Adding query filter params (role.id=ca13aabb28b6442f8540fa4e8bb1b35a, scope.project.id=3f0bb97051994c89b3544850957b48be, user.id=7563b6e0ea29491291eb70a0be54b165) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:23.603390 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42bd5a-1db6-4512-a437-7ca71bc6ce52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:23.603390 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42bd5a-1db6-4512-a437-7ca71bc6ce52 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:23.625481 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:23.629294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 46/95] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:23 2022] GET /identity/v3/role_assignments?user.id=7563b6e0ea29491291eb70a0be54b165&scope.project.id=3f0bb97051994c89b3544850957b48be&role.id=ca13aabb28b6442f8540fa4e8bb1b35a => generated 259 bytes in 597 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:23.705465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a58711-9867-493f-8ecd-f810258e8d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:23.707071 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a58711-9867-493f-8ecd-f810258e8d17 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:23.707881 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a58711-9867-493f-8ecd-f810258e8d17 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:23.708695 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a58711-9867-493f-8ecd-f810258e8d17 None admin] PATH_INFO: `/v3/projects/48446be76b3048ebba9328ab935f4256` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:23.711215 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a58711-9867-493f-8ecd-f810258e8d17 None admin] RBAC: Authorizing `identity:get_project(project_id=48446be76b3048ebba9328ab935f4256)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:23.720280 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a58711-9867-493f-8ecd-f810258e8d17 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:23.724070 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 50/96] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:23 2022] GET /identity/v3/projects/48446be76b3048ebba9328ab935f4256 => generated 312 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:23.734836 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42bd5a-1db6-4512-a437-7ca71bc6ce52 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:24.000676 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42bd5a-1db6-4512-a437-7ca71bc6ce52 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:24.067144 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e75bbe-1c78-40b7-91fb-5450a072c61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:24.068196 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e75bbe-1c78-40b7-91fb-5450a072c61e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:24.068690 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e75bbe-1c78-40b7-91fb-5450a072c61e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:24.069187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e75bbe-1c78-40b7-91fb-5450a072c61e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:24.070602 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e75bbe-1c78-40b7-91fb-5450a072c61e None admin] RBAC: Adding query filter params (role.id=d18e4897bf3648498a472a2ea8174f8b, scope.project.id=48446be76b3048ebba9328ab935f4256, user.id=7563b6e0ea29491291eb70a0be54b165) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:24.071166 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e75bbe-1c78-40b7-91fb-5450a072c61e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:24.073912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e75bbe-1c78-40b7-91fb-5450a072c61e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:24.085056 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 47/97] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:23 2022] GET /identity/v3/role_assignments?user.id=7563b6e0ea29491291eb70a0be54b165&scope.project.id=48446be76b3048ebba9328ab935f4256&role.id=d18e4897bf3648498a472a2ea8174f8b => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:25.131860 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac39d04-33be-4f9b-9517-1dde714c6e31 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:25.133488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac39d04-33be-4f9b-9517-1dde714c6e31 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:25.134313 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac39d04-33be-4f9b-9517-1dde714c6e31 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:25.137757 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 51/98] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:25.145383 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f3994-0bf0-440f-ba6b-71431f755053 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:25.146938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f3994-0bf0-440f-ba6b-71431f755053 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:25.147682 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f3994-0bf0-440f-ba6b-71431f755053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:25.183254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-cc9f3994-0bf0-440f-ba6b-71431f755053 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:25.185814 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cc9f3994-0bf0-440f-ba6b-71431f755053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:25.300430 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:25.343762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 48/99] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:25.878619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b5cc5-65c1-4380-888a-b5693b999c34 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:25.879745 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b5cc5-65c1-4380-888a-b5693b999c34 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:25.880549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b5cc5-65c1-4380-888a-b5693b999c34 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:25.883620 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 52/100] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:25.893317 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df5037-bdd0-446a-9362-293c88bdc756 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:25.894387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df5037-bdd0-446a-9362-293c88bdc756 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:25.895497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df5037-bdd0-446a-9362-293c88bdc756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:25.940053 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-75df5037-bdd0-446a-9362-293c88bdc756 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:25.942785 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-75df5037-bdd0-446a-9362-293c88bdc756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:26.077500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 49/101] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:26.135096 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b5cc5-65c1-4380-888a-b5693b999c34 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:26.245198 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d68ae-bd81-4e63-988f-3ddead0e2820 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:26.246030 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d68ae-bd81-4e63-988f-3ddead0e2820 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:26.246030 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d68ae-bd81-4e63-988f-3ddead0e2820 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:26.247209 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 50/102] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:26.254105 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a06ef0-a74f-4147-b5fa-81cedaccbb07 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:26.254105 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a06ef0-a74f-4147-b5fa-81cedaccbb07 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:26.254105 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a06ef0-a74f-4147-b5fa-81cedaccbb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:26.307380 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-10a06ef0-a74f-4147-b5fa-81cedaccbb07 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:26.308624 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-10a06ef0-a74f-4147-b5fa-81cedaccbb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:26.424335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 51/103] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:26.484588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b5cc5-65c1-4380-888a-b5693b999c34 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:26.546824 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67ec30-1f31-4b4e-9511-c7329691f37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:26.547477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67ec30-1f31-4b4e-9511-c7329691f37e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:26.547760 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67ec30-1f31-4b4e-9511-c7329691f37e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:26.548038 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67ec30-1f31-4b4e-9511-c7329691f37e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:26.554755 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67ec30-1f31-4b4e-9511-c7329691f37e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:26.556250 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67ec30-1f31-4b4e-9511-c7329691f37e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:26.557880 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 53/104] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:26.563607 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a06ef0-a74f-4147-b5fa-81cedaccbb07 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:26.676707 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:26.791415 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a06ef0-a74f-4147-b5fa-81cedaccbb07 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:26.882557 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4daa8-ed68-45d3-ac41-1224cc92b8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:26.884153 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4daa8-ed68-45d3-ac41-1224cc92b8c5 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:26.884909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4daa8-ed68-45d3-ac41-1224cc92b8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:26.885654 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4daa8-ed68-45d3-ac41-1224cc92b8c5 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:26.887827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb4daa8-ed68-45d3-ac41-1224cc92b8c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:26.891002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb4daa8-ed68-45d3-ac41-1224cc92b8c5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:26.921216 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67ec30-1f31-4b4e-9511-c7329691f37e None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:26.940699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 52/105] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:34:26 2022] POST /identity/v3/projects => generated 302 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:34:26.941292 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4daa8-ed68-45d3-ac41-1224cc92b8c5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:27.066777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:27.082513 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:27.100652 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4daa8-ed68-45d3-ac41-1224cc92b8c5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:27.199094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f12d8-aaed-4279-b712-e7409ca141ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:27.200285 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f12d8-aaed-4279-b712-e7409ca141ba None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:27.200577 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f12d8-aaed-4279-b712-e7409ca141ba None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:27.200842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f12d8-aaed-4279-b712-e7409ca141ba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:27.235465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67ec30-1f31-4b4e-9511-c7329691f37e None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:27.250715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f12d8-aaed-4279-b712-e7409ca141ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-df5f12d8-aaed-4279-b712-e7409ca141ba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:27.256247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:27.258003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:27.259754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 53/106] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:27.267209 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f12d8-aaed-4279-b712-e7409ca141ba None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:27.374622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63065d-d639-41d4-88b8-6def8a139b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:27.376292 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63065d-d639-41d4-88b8-6def8a139b36 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:27.377108 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63065d-d639-41d4-88b8-6def8a139b36 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:27.377886 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63065d-d639-41d4-88b8-6def8a139b36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:27.400734 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f12d8-aaed-4279-b712-e7409ca141ba None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:27.411785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63065d-d639-41d4-88b8-6def8a139b36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-bd63065d-d639-41d4-88b8-6def8a139b36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:27.425401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:27.427188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:27.428765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:27.428765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:27.428765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:27.428765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:27.428765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:27.428765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:27.428765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:27.428765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:27.428765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:27.428765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:27.428765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:27.428765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:27.433207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 54/107] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:34:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 513 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:34:27.436510 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff54a13-bbc8-440f-9ec1-28501cf1d415 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:27.437340 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff54a13-bbc8-440f-9ec1-28501cf1d415 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:27.446113 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acccf79-0e45-48ca-a011-623deaadee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:27.446113 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acccf79-0e45-48ca-a011-623deaadee77 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:27.446113 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acccf79-0e45-48ca-a011-623deaadee77 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:27.446113 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acccf79-0e45-48ca-a011-623deaadee77 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:27.446720 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acccf79-0e45-48ca-a011-623deaadee77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:27.446720 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acccf79-0e45-48ca-a011-623deaadee77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:27.446720 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acccf79-0e45-48ca-a011-623deaadee77 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:27.447595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff54a13-bbc8-440f-9ec1-28501cf1d415 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:27.450887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 55/108] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:27 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:27.453664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 54/109] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:27.454118 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff54a13-bbc8-440f-9ec1-28501cf1d415 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:27.457820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbeb5e-f069-40d4-8b7a-9706afdc8fa3 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:27.458228 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbeb5e-f069-40d4-8b7a-9706afdc8fa3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:27.458393 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbeb5e-f069-40d4-8b7a-9706afdc8fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:27.493060 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f7bbeb5e-f069-40d4-8b7a-9706afdc8fa3 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:27.495781 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f7bbeb5e-f069-40d4-8b7a-9706afdc8fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:27.530547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:27.646704 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 55/110] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:27.648211 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbeb5e-f069-40d4-8b7a-9706afdc8fa3 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:27.775458 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff54a13-bbc8-440f-9ec1-28501cf1d415 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:27.788997 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:27.842109 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbeb5e-f069-40d4-8b7a-9706afdc8fa3 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:27.858937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a0ddf-7013-4274-9d74-b866dbd84d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:27.859986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a0ddf-7013-4274-9d74-b866dbd84d7b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:27.860497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a0ddf-7013-4274-9d74-b866dbd84d7b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:27.860971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a0ddf-7013-4274-9d74-b866dbd84d7b None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:27.861902 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a0ddf-7013-4274-9d74-b866dbd84d7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:27.862457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a0ddf-7013-4274-9d74-b866dbd84d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:27.864159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a0ddf-7013-4274-9d74-b866dbd84d7b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:27.893782 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 56/111] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:27.896310 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f06f71-4f78-4abf-aa89-32b1869bbcd2 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:27.896807 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f06f71-4f78-4abf-aa89-32b1869bbcd2 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:27.897303 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f06f71-4f78-4abf-aa89-32b1869bbcd2 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:27.899700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 57/112] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:27.906196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f06f71-4f78-4abf-aa89-32b1869bbcd2 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:27.971919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb34fe2-54fe-4be3-ae19-18dbc00dec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:27.972984 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb34fe2-54fe-4be3-ae19-18dbc00dec79 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:27.973471 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb34fe2-54fe-4be3-ae19-18dbc00dec79 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:27.973969 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb34fe2-54fe-4be3-ae19-18dbc00dec79 None admin] PATH_INFO: `/v3/users/bbc83d86dc05486faa3364a168ae610d` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:27.993034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb34fe2-54fe-4be3-ae19-18dbc00dec79 None admin] RBAC: Authorizing `identity:get_user(user_id=bbc83d86dc05486faa3364a168ae610d)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:27.995082 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb34fe2-54fe-4be3-ae19-18dbc00dec79 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:27.997267 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 56/113] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:27 2022] GET /identity/v3/users/bbc83d86dc05486faa3364a168ae610d => generated 289 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:27.998893 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefb114-8c0d-45a3-bf6d-3b4f1802297d None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:27.999419 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefb114-8c0d-45a3-bf6d-3b4f1802297d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:27.999916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefb114-8c0d-45a3-bf6d-3b4f1802297d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:28.020617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3aefb114-8c0d-45a3-bf6d-3b4f1802297d None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:28.023029 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:28.024536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3aefb114-8c0d-45a3-bf6d-3b4f1802297d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:28.110826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 57/114] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:28.112172 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefb114-8c0d-45a3-bf6d-3b4f1802297d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:28.149113 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f06f71-4f78-4abf-aa89-32b1869bbcd2 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:28.219564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4ed1f-1c6f-439e-815f-c6efb14811f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:28.220402 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4ed1f-1c6f-439e-815f-c6efb14811f5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:28.220895 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4ed1f-1c6f-439e-815f-c6efb14811f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:28.221372 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4ed1f-1c6f-439e-815f-c6efb14811f5 None admin] PATH_INFO: `/v3/users/ba3be3b6260847d081e4ed3e9d15d818` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:28.223658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4ed1f-1c6f-439e-815f-c6efb14811f5 None admin] RBAC: Authorizing `identity:get_user(user_id=ba3be3b6260847d081e4ed3e9d15d818)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:28.226494 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4ed1f-1c6f-439e-815f-c6efb14811f5 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:28.229674 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 58/115] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:27 2022] GET /identity/v3/users/ba3be3b6260847d081e4ed3e9d15d818 => generated 247 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:28.231416 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4ed1f-1c6f-439e-815f-c6efb14811f5 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:28.236771 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefb114-8c0d-45a3-bf6d-3b4f1802297d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:28.279974 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8673a-f53a-4c40-a25e-152bff0ce1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:28.281487 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8673a-f53a-4c40-a25e-152bff0ce1ea None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:28.282164 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8673a-f53a-4c40-a25e-152bff0ce1ea None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:28.282639 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8673a-f53a-4c40-a25e-152bff0ce1ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:28.286149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:28.297442 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8673a-f53a-4c40-a25e-152bff0ce1ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-d0a8673a-f53a-4c40-a25e-152bff0ce1ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:28.303133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:28.304002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:28.304780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:28.304780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:28.304780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:28.304780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:28.304780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:28.304780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:28.304780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:28.304780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:28.304780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:28.304780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:28.304780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:28.304780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:28.307309 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 58/116] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:28.308451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8673a-f53a-4c40-a25e-152bff0ce1ea None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:28.312426 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:28.432690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4ed1f-1c6f-439e-815f-c6efb14811f5 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:28.497916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8673a-f53a-4c40-a25e-152bff0ce1ea None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:28.507152 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17f04f-4bac-484f-9768-a9872addc86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:28.508202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17f04f-4bac-484f-9768-a9872addc86b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:28.508683 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17f04f-4bac-484f-9768-a9872addc86b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:28.509157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17f04f-4bac-484f-9768-a9872addc86b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:28.570087 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd148b-5d9f-4d47-a56b-87e066d5b17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:28.571844 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd148b-5d9f-4d47-a56b-87e066d5b17e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:28.572642 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd148b-5d9f-4d47-a56b-87e066d5b17e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:28.573448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd148b-5d9f-4d47-a56b-87e066d5b17e None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:28.576594 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd148b-5d9f-4d47-a56b-87e066d5b17e None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:28.579790 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd148b-5d9f-4d47-a56b-87e066d5b17e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:28.583362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 59/117] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:28 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:28.585215 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd148b-5d9f-4d47-a56b-87e066d5b17e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:28.646847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17f04f-4bac-484f-9768-a9872addc86b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:28.649654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17f04f-4bac-484f-9768-a9872addc86b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:28.703695 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:28.706568 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:28.753065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd148b-5d9f-4d47-a56b-87e066d5b17e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:28.795553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 59/118] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:28.796646 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94665214-5f6a-4e78-bc37-b115e041a591 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:28.796922 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94665214-5f6a-4e78-bc37-b115e041a591 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:28.797193 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94665214-5f6a-4e78-bc37-b115e041a591 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:28.798731 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 60/119] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:28.799302 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94665214-5f6a-4e78-bc37-b115e041a591 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:28.810970 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7aa2d-770a-4f03-b5f0-69e7896abefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:28.811614 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7aa2d-770a-4f03-b5f0-69e7896abefa None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:28.811971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7aa2d-770a-4f03-b5f0-69e7896abefa None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:28.812254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7aa2d-770a-4f03-b5f0-69e7896abefa None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:28.812946 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7aa2d-770a-4f03-b5f0-69e7896abefa None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:28.813253 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7aa2d-770a-4f03-b5f0-69e7896abefa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:28.814699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7aa2d-770a-4f03-b5f0-69e7896abefa None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:28.823851 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 60/120] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:28.825301 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7aa2d-770a-4f03-b5f0-69e7896abefa None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:28.938333 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7aa2d-770a-4f03-b5f0-69e7896abefa None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:28.968999 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:28.980173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a46de-7cbb-44a1-a209-2f604e42a481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:28.980950 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a46de-7cbb-44a1-a209-2f604e42a481 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:28.981237 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a46de-7cbb-44a1-a209-2f604e42a481 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:28.981513 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a46de-7cbb-44a1-a209-2f604e42a481 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:28.988480 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94665214-5f6a-4e78-bc37-b115e041a591 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:28.995461 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a46de-7cbb-44a1-a209-2f604e42a481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-201a46de-7cbb-44a1-a209-2f604e42a481 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:29.001040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:29.001963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:29.002753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 61/121] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:34:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:29.008488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a46de-7cbb-44a1-a209-2f604e42a481 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:29.043640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a2b5c-36d8-44fe-b7c3-9c393aa0aa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:29.044322 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a2b5c-36d8-44fe-b7c3-9c393aa0aa9d None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:29.044599 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a2b5c-36d8-44fe-b7c3-9c393aa0aa9d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:29.045040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a2b5c-36d8-44fe-b7c3-9c393aa0aa9d None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be/users/ba3be3b6260847d081e4ed3e9d15d818/roles/fc68d072459d4720b9666c93bd535d63` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:29.047019 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a2b5c-36d8-44fe-b7c3-9c393aa0aa9d None admin] RBAC: Authorizing `identity:create_grant(project_id=3f0bb97051994c89b3544850957b48be, user_id=ba3be3b6260847d081e4ed3e9d15d818, role_id=fc68d072459d4720b9666c93bd535d63)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:29.048732 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a2b5c-36d8-44fe-b7c3-9c393aa0aa9d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:29.052121 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:29.067058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 61/122] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:34:28 2022] PUT /identity/v3/projects/3f0bb97051994c89b3544850957b48be/users/ba3be3b6260847d081e4ed3e9d15d818/roles/fc68d072459d4720b9666c93bd535d63 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:34:29.068550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a02658-eae1-4483-b7b5-4e5c4ef66cd0 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:29.068817 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a02658-eae1-4483-b7b5-4e5c4ef66cd0 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:29.069075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a02658-eae1-4483-b7b5-4e5c4ef66cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:29.097693 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-02a02658-eae1-4483-b7b5-4e5c4ef66cd0 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:29.099923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-02a02658-eae1-4483-b7b5-4e5c4ef66cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:29.172950 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a46de-7cbb-44a1-a209-2f604e42a481 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:29.233661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffe6bb-d778-4738-95d2-8c87ac418ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:29.234986 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe6bb-d778-4738-95d2-8c87ac418ca5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:29.235534 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe6bb-d778-4738-95d2-8c87ac418ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:29.236081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe6bb-d778-4738-95d2-8c87ac418ca5 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:29.239140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffe6bb-d778-4738-95d2-8c87ac418ca5 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:29.242281 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffe6bb-d778-4738-95d2-8c87ac418ca5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:29.244675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 62/123] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:29.245320 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 62/124] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:29 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:29.247104 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a02658-eae1-4483-b7b5-4e5c4ef66cd0 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:29.275789 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffe6bb-d778-4738-95d2-8c87ac418ca5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:29.426731 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:29.499842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a02658-eae1-4483-b7b5-4e5c4ef66cd0 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:29.535282 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffe6bb-d778-4738-95d2-8c87ac418ca5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:29.556795 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:29.581300 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f909361-8875-4bf0-9ae6-02e612393baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:29.582844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f909361-8875-4bf0-9ae6-02e612393baf None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:29.583355 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f909361-8875-4bf0-9ae6-02e612393baf None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:29.583856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f909361-8875-4bf0-9ae6-02e612393baf None admin] PATH_INFO: `/v3/users/7563b6e0ea29491291eb70a0be54b165` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:29.590305 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f909361-8875-4bf0-9ae6-02e612393baf None admin] RBAC: Authorizing `identity:get_user(user_id=7563b6e0ea29491291eb70a0be54b165)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:29.590305 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f909361-8875-4bf0-9ae6-02e612393baf None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:29.594287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 63/125] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:29 2022] GET /identity/v3/users/7563b6e0ea29491291eb70a0be54b165 => generated 275 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:29.596066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f909361-8875-4bf0-9ae6-02e612393baf None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:29.669467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bfc4a-e442-4288-9adf-55dbfc188cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:29.671146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bfc4a-e442-4288-9adf-55dbfc188cf2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:29.671919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bfc4a-e442-4288-9adf-55dbfc188cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:29.672608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bfc4a-e442-4288-9adf-55dbfc188cf2 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:29.674178 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329bfc4a-e442-4288-9adf-55dbfc188cf2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:29.675003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329bfc4a-e442-4288-9adf-55dbfc188cf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:29.686083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329bfc4a-e442-4288-9adf-55dbfc188cf2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:29.718513 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 63/126] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:34:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:29.720412 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bfc4a-e442-4288-9adf-55dbfc188cf2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:29.834570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f909361-8875-4bf0-9ae6-02e612393baf None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:29.923752 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4abdf-b59b-4aef-a827-b0d26234ec46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:29.924874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4abdf-b59b-4aef-a827-b0d26234ec46 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:29.925580 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4abdf-b59b-4aef-a827-b0d26234ec46 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:29.926273 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4abdf-b59b-4aef-a827-b0d26234ec46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:29.927621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4abdf-b59b-4aef-a827-b0d26234ec46 None admin] RBAC: Adding query filter params (role.id=1b0fe0772e714d1ba454d992ef8fa266, scope.project.id=3f0bb97051994c89b3544850957b48be, user.id=bbc83d86dc05486faa3364a168ae610d) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:29.928156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4abdf-b59b-4aef-a827-b0d26234ec46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:29.932217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4abdf-b59b-4aef-a827-b0d26234ec46 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:29.946085 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 64/127] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:29 2022] GET /identity/v3/role_assignments?user.id=bbc83d86dc05486faa3364a168ae610d&scope.project.id=3f0bb97051994c89b3544850957b48be&role.id=1b0fe0772e714d1ba454d992ef8fa266 => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:29.947888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4abdf-b59b-4aef-a827-b0d26234ec46 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:29.969439 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bfc4a-e442-4288-9adf-55dbfc188cf2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:30.014988 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:30.082458 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:30.083785 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f0f1f-b444-468c-a949-dd569e1ef277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:30.084971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f0f1f-b444-468c-a949-dd569e1ef277 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:30.085393 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f0f1f-b444-468c-a949-dd569e1ef277 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:30.085879 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f0f1f-b444-468c-a949-dd569e1ef277 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:30.094311 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f0f1f-b444-468c-a949-dd569e1ef277 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:30.097433 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f0f1f-b444-468c-a949-dd569e1ef277 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:30.100430 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 64/128] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:29 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:30.101883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f0f1f-b444-468c-a949-dd569e1ef277 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:30.179819 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4abdf-b59b-4aef-a827-b0d26234ec46 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:30.248225 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbf6de-c7f5-4be4-894e-1f73eb1cf4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:30.249838 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbf6de-c7f5-4be4-894e-1f73eb1cf4cc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:30.250638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbf6de-c7f5-4be4-894e-1f73eb1cf4cc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:30.251371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbf6de-c7f5-4be4-894e-1f73eb1cf4cc None admin] PATH_INFO: `/v3/users/7563b6e0ea29491291eb70a0be54b165` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:30.253982 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbf6de-c7f5-4be4-894e-1f73eb1cf4cc None admin] RBAC: Authorizing `identity:get_user(user_id=7563b6e0ea29491291eb70a0be54b165)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:30.257224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbf6de-c7f5-4be4-894e-1f73eb1cf4cc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:30.261264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 65/129] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:29 2022] GET /identity/v3/users/7563b6e0ea29491291eb70a0be54b165 => generated 275 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:30.263163 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbf6de-c7f5-4be4-894e-1f73eb1cf4cc None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:30.415215 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f0f1f-b444-468c-a949-dd569e1ef277 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:30.484341 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbf6de-c7f5-4be4-894e-1f73eb1cf4cc None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:30.546214 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748702f-dc3a-4f3e-b2da-71d4898634c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:30.547334 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748702f-dc3a-4f3e-b2da-71d4898634c9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:30.547820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748702f-dc3a-4f3e-b2da-71d4898634c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:30.548301 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748702f-dc3a-4f3e-b2da-71d4898634c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:30.568952 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:30.582059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748702f-dc3a-4f3e-b2da-71d4898634c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0748702f-dc3a-4f3e-b2da-71d4898634c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:30.589784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:30.590855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:30.591728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:30.591728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:30.591728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:30.591728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:30.591728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:30.591728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:30.591728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:30.591728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:30.591728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:30.591728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:30.591728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:30.591728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:30.597290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 65/130] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 496 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:30.598314 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d31b7-8db4-42ff-bd21-f0a0d1ec727f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:30.599269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d31b7-8db4-42ff-bd21-f0a0d1ec727f None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:30.599716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d31b7-8db4-42ff-bd21-f0a0d1ec727f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:30.600129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d31b7-8db4-42ff-bd21-f0a0d1ec727f None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be/users/7563b6e0ea29491291eb70a0be54b165/roles/d18e4897bf3648498a472a2ea8174f8b` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:30.602181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748702f-dc3a-4f3e-b2da-71d4898634c9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:30.604129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d31b7-8db4-42ff-bd21-f0a0d1ec727f None admin] RBAC: Authorizing `identity:create_grant(project_id=3f0bb97051994c89b3544850957b48be, user_id=7563b6e0ea29491291eb70a0be54b165, role_id=d18e4897bf3648498a472a2ea8174f8b)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:30.606458 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d31b7-8db4-42ff-bd21-f0a0d1ec727f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:30.641619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 66/131] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:34:30 2022] PUT /identity/v3/projects/3f0bb97051994c89b3544850957b48be/users/7563b6e0ea29491291eb70a0be54b165/roles/d18e4897bf3648498a472a2ea8174f8b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:34:30.643345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d31b7-8db4-42ff-bd21-f0a0d1ec727f None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:30.665127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:30.770530 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748702f-dc3a-4f3e-b2da-71d4898634c9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:30.832012 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864514b-e03b-4b9d-a2b4-2482d7dc46f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:30.832830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864514b-e03b-4b9d-a2b4-2482d7dc46f1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:30.833213 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864514b-e03b-4b9d-a2b4-2482d7dc46f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:30.833588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864514b-e03b-4b9d-a2b4-2482d7dc46f1 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:30.834973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864514b-e03b-4b9d-a2b4-2482d7dc46f1 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:30.837011 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864514b-e03b-4b9d-a2b4-2482d7dc46f1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:30.838834 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 66/132] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:30 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:30.848469 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864514b-e03b-4b9d-a2b4-2482d7dc46f1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:30.873735 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d31b7-8db4-42ff-bd21-f0a0d1ec727f None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:30.950140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c630403-c5ce-4be6-b081-2e87d6242992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:30.951793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630403-c5ce-4be6-b081-2e87d6242992 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:30.952573 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630403-c5ce-4be6-b081-2e87d6242992 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:30.953484 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c630403-c5ce-4be6-b081-2e87d6242992 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:30.955075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c630403-c5ce-4be6-b081-2e87d6242992 None admin] RBAC: Adding query filter params (name=member) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:30.956014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c630403-c5ce-4be6-b081-2e87d6242992 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:30.959099 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c630403-c5ce-4be6-b081-2e87d6242992 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:30.978012 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 67/133] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:30.988870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c630403-c5ce-4be6-b081-2e87d6242992 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:31.033661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864514b-e03b-4b9d-a2b4-2482d7dc46f1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:31.057070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:31.097480 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:31.103940 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29dc79-ebc0-487e-b1ff-b9851c947701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:31.104660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29dc79-ebc0-487e-b1ff-b9851c947701 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:31.104957 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29dc79-ebc0-487e-b1ff-b9851c947701 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:31.105207 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29dc79-ebc0-487e-b1ff-b9851c947701 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be/users/7563b6e0ea29491291eb70a0be54b165/roles/ca13aabb28b6442f8540fa4e8bb1b35a` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:31.107623 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29dc79-ebc0-487e-b1ff-b9851c947701 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f0bb97051994c89b3544850957b48be, user_id=7563b6e0ea29491291eb70a0be54b165, role_id=ca13aabb28b6442f8540fa4e8bb1b35a)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:31.109095 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29dc79-ebc0-487e-b1ff-b9851c947701 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:31.157045 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 67/134] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:34:30 2022] PUT /identity/v3/projects/3f0bb97051994c89b3544850957b48be/users/7563b6e0ea29491291eb70a0be54b165/roles/ca13aabb28b6442f8540fa4e8bb1b35a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:34:31.179792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956b988-1883-4753-b8d2-1a916676689a None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:31.180237 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956b988-1883-4753-b8d2-1a916676689a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:31.180406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956b988-1883-4753-b8d2-1a916676689a None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:31.181759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 68/135] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:34:31.195595 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08a8f9-7974-489d-aec6-2a793f01335e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:31.195862 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08a8f9-7974-489d-aec6-2a793f01335e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:31.196207 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08a8f9-7974-489d-aec6-2a793f01335e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:31.226367 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c630403-c5ce-4be6-b081-2e87d6242992 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:31.230325 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-df08a8f9-7974-489d-aec6-2a793f01335e None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:31.235926 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-df08a8f9-7974-489d-aec6-2a793f01335e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:31.305125 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135278b0-feba-4e98-af4d-22ce28f67974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:31.306688 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135278b0-feba-4e98-af4d-22ce28f67974 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:31.307552 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135278b0-feba-4e98-af4d-22ce28f67974 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:31.309084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135278b0-feba-4e98-af4d-22ce28f67974 None admin] PATH_INFO: `/v3/users/7563b6e0ea29491291eb70a0be54b165` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:31.311802 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135278b0-feba-4e98-af4d-22ce28f67974 None admin] RBAC: Authorizing `identity:get_user(user_id=7563b6e0ea29491291eb70a0be54b165)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:31.315114 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135278b0-feba-4e98-af4d-22ce28f67974 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:31.321055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 68/136] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:30 2022] GET /identity/v3/users/7563b6e0ea29491291eb70a0be54b165 => generated 275 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:31.329370 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135278b0-feba-4e98-af4d-22ce28f67974 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:31.477906 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 69/137] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:31.533311 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135278b0-feba-4e98-af4d-22ce28f67974 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:31.582733 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:31.612165 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f1e4a-53b3-4af3-84d8-32d357bb0383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:31.614054 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f1e4a-53b3-4af3-84d8-32d357bb0383 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:31.614563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f1e4a-53b3-4af3-84d8-32d357bb0383 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:31.615048 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f1e4a-53b3-4af3-84d8-32d357bb0383 None admin] PATH_INFO: `/v3/projects/48446be76b3048ebba9328ab935f4256` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:31.617369 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f1e4a-53b3-4af3-84d8-32d357bb0383 None admin] RBAC: Authorizing `identity:get_project(project_id=48446be76b3048ebba9328ab935f4256)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:31.620207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f1e4a-53b3-4af3-84d8-32d357bb0383 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:31.623664 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 69/138] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:31 2022] GET /identity/v3/projects/48446be76b3048ebba9328ab935f4256 => generated 312 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:31.632995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08a8f9-7974-489d-aec6-2a793f01335e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:31.858774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08a8f9-7974-489d-aec6-2a793f01335e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:31.861103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:31.910689 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d013a9-ab49-4206-8a66-b0bd4bf737d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:31.911374 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d013a9-ab49-4206-8a66-b0bd4bf737d0 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:31.911658 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d013a9-ab49-4206-8a66-b0bd4bf737d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:31.911928 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d013a9-ab49-4206-8a66-b0bd4bf737d0 None admin] PATH_INFO: `/v3/projects/48446be76b3048ebba9328ab935f4256/users/7563b6e0ea29491291eb70a0be54b165/roles/d18e4897bf3648498a472a2ea8174f8b` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:31.913922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d013a9-ab49-4206-8a66-b0bd4bf737d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=48446be76b3048ebba9328ab935f4256, user_id=7563b6e0ea29491291eb70a0be54b165, role_id=d18e4897bf3648498a472a2ea8174f8b)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:31.915426 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d013a9-ab49-4206-8a66-b0bd4bf737d0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:31.931179 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 70/139] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:34:31 2022] PUT /identity/v3/projects/48446be76b3048ebba9328ab935f4256/users/7563b6e0ea29491291eb70a0be54b165/roles/d18e4897bf3648498a472a2ea8174f8b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:34:31.939635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f1e4a-53b3-4af3-84d8-32d357bb0383 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:32.151093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f1e4a-53b3-4af3-84d8-32d357bb0383 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:32.220444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0ac0a-e80d-466e-a0ec-c781bbab9775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:32.221980 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0ac0a-e80d-466e-a0ec-c781bbab9775 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:32.222820 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0ac0a-e80d-466e-a0ec-c781bbab9775 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:32.223561 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0ac0a-e80d-466e-a0ec-c781bbab9775 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:32.225677 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0ac0a-e80d-466e-a0ec-c781bbab9775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:32.228716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0ac0a-e80d-466e-a0ec-c781bbab9775 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:32.232316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 70/140] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:32.238100 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d013a9-ab49-4206-8a66-b0bd4bf737d0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:32.470536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d013a9-ab49-4206-8a66-b0bd4bf737d0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:32.535729 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e2f55-91c8-441b-9035-112ccfb1703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:32.536743 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e2f55-91c8-441b-9035-112ccfb1703c None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:32.537221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e2f55-91c8-441b-9035-112ccfb1703c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:32.537701 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e2f55-91c8-441b-9035-112ccfb1703c None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:32.538934 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e2f55-91c8-441b-9035-112ccfb1703c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:32.541761 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e2f55-91c8-441b-9035-112ccfb1703c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:32.578946 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 71/141] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:34:32 2022] POST /identity/v3/users => generated 283 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:34:33.052362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93507735-6b21-4405-a234-2f1c30746107 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:33.053863 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93507735-6b21-4405-a234-2f1c30746107 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:33.054633 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93507735-6b21-4405-a234-2f1c30746107 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:33.057860 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 71/142] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:33.064597 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f5e27-91a0-4c40-81a2-1971456eef26 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:33.065986 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f5e27-91a0-4c40-81a2-1971456eef26 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:33.066787 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f5e27-91a0-4c40-81a2-1971456eef26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:33.110791 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-812f5e27-91a0-4c40-81a2-1971456eef26 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:33.113399 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-812f5e27-91a0-4c40-81a2-1971456eef26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:33.150305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:33.273052 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 72/143] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:34:33.814539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93507735-6b21-4405-a234-2f1c30746107 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:33.918320 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:34.023318 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93507735-6b21-4405-a234-2f1c30746107 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:34.088468 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c8356-fd73-43cb-abbd-b93323a58436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:34.089397 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c8356-fd73-43cb-abbd-b93323a58436 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:34.090069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c8356-fd73-43cb-abbd-b93323a58436 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:34.090645 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c8356-fd73-43cb-abbd-b93323a58436 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:34.115463 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c8356-fd73-43cb-abbd-b93323a58436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-c89c8356-fd73-43cb-abbd-b93323a58436 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:34.127415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:34.129100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:34.130542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 72/144] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:34:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:34.139344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f5e27-91a0-4c40-81a2-1971456eef26 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:34.359393 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f5e27-91a0-4c40-81a2-1971456eef26 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:34.430450 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32371f10-c2ba-4d80-bfa4-9ee475edfbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:34.431628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32371f10-c2ba-4d80-bfa4-9ee475edfbb4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:34.432038 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32371f10-c2ba-4d80-bfa4-9ee475edfbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:34.432323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32371f10-c2ba-4d80-bfa4-9ee475edfbb4 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:34.433050 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32371f10-c2ba-4d80-bfa4-9ee475edfbb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:34.433967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32371f10-c2ba-4d80-bfa4-9ee475edfbb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:34.435388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32371f10-c2ba-4d80-bfa4-9ee475edfbb4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:34.454637 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 73/145] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:34.460517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c8356-fd73-43cb-abbd-b93323a58436 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:34.583913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c8356-fd73-43cb-abbd-b93323a58436 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:34.611222 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:34.625109 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bc380-8db5-4951-98d2-c52dafabb07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:34.625739 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bc380-8db5-4951-98d2-c52dafabb07f None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:34.626020 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bc380-8db5-4951-98d2-c52dafabb07f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:34.626322 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bc380-8db5-4951-98d2-c52dafabb07f None admin] PATH_INFO: `/v3/users/ba3be3b6260847d081e4ed3e9d15d818` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:34.627543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bc380-8db5-4951-98d2-c52dafabb07f None admin] RBAC: Authorizing `identity:get_user(user_id=ba3be3b6260847d081e4ed3e9d15d818)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:34.629036 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bc380-8db5-4951-98d2-c52dafabb07f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:34.631010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 73/146] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:34 2022] GET /identity/v3/users/ba3be3b6260847d081e4ed3e9d15d818 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:34.637022 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32371f10-c2ba-4d80-bfa4-9ee475edfbb4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:34.653478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:34.766427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32371f10-c2ba-4d80-bfa4-9ee475edfbb4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:34.812443 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0973b5-e0e9-4993-9cc8-88cb666b0f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:34.813081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0973b5-e0e9-4993-9cc8-88cb666b0f4e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:34.813357 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0973b5-e0e9-4993-9cc8-88cb666b0f4e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:34.813628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0973b5-e0e9-4993-9cc8-88cb666b0f4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:34.827564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5962aaa-5e44-48ed-861a-3b8004be67d3 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:34.827858 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5962aaa-5e44-48ed-861a-3b8004be67d3 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:34.828174 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5962aaa-5e44-48ed-861a-3b8004be67d3 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:34.829856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 74/147] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:34:34.854290 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927c8d9-b59a-4f83-bc9b-06647772e9a0 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:34.854691 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927c8d9-b59a-4f83-bc9b-06647772e9a0 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:34.854824 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927c8d9-b59a-4f83-bc9b-06647772e9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:34.897354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-1927c8d9-b59a-4f83-bc9b-06647772e9a0 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:34.898569 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-1927c8d9-b59a-4f83-bc9b-06647772e9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:34.990258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0973b5-e0e9-4993-9cc8-88cb666b0f4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:34.991807 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0973b5-e0e9-4993-9cc8-88cb666b0f4e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:35.037481 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:35.055268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 75/148] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:35.319978 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 74/149] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:35.326800 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927c8d9-b59a-4f83-bc9b-06647772e9a0 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:35.539476 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927c8d9-b59a-4f83-bc9b-06647772e9a0 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:35.557936 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:35.619452 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a659b02-d4b9-44d5-9f37-0439484cc9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:35.621081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a659b02-d4b9-44d5-9f37-0439484cc9ec None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:35.621903 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a659b02-d4b9-44d5-9f37-0439484cc9ec None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:35.622696 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a659b02-d4b9-44d5-9f37-0439484cc9ec None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:35.625115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a659b02-d4b9-44d5-9f37-0439484cc9ec None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:35.628109 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a659b02-d4b9-44d5-9f37-0439484cc9ec None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:35.631632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 76/150] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:35 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:35.638794 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0973b5-e0e9-4993-9cc8-88cb666b0f4e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:35.929248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0973b5-e0e9-4993-9cc8-88cb666b0f4e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:35.960936 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:36.014997 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681cb75-a5e3-4d21-a5c3-e06bbf7a14a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:36.016021 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681cb75-a5e3-4d21-a5c3-e06bbf7a14a5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:36.016497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681cb75-a5e3-4d21-a5c3-e06bbf7a14a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:36.016971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681cb75-a5e3-4d21-a5c3-e06bbf7a14a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:36.018382 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681cb75-a5e3-4d21-a5c3-e06bbf7a14a5 None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=3f0bb97051994c89b3544850957b48be, user.id=ba3be3b6260847d081e4ed3e9d15d818) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:36.018911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681cb75-a5e3-4d21-a5c3-e06bbf7a14a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:36.021893 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681cb75-a5e3-4d21-a5c3-e06bbf7a14a5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:36.038362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 75/151] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:35 2022] GET /identity/v3/role_assignments?user.id=ba3be3b6260847d081e4ed3e9d15d818&scope.project.id=3f0bb97051994c89b3544850957b48be&role.id=fc68d072459d4720b9666c93bd535d63 => generated 608 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:36.134527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a659b02-d4b9-44d5-9f37-0439484cc9ec None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:36.287356 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a659b02-d4b9-44d5-9f37-0439484cc9ec None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:36.332408 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f861559-c2f4-46e9-a6f9-7853724f04a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:36.333366 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f861559-c2f4-46e9-a6f9-7853724f04a5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:36.333366 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f861559-c2f4-46e9-a6f9-7853724f04a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:36.333591 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f861559-c2f4-46e9-a6f9-7853724f04a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:36.349518 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f861559-c2f4-46e9-a6f9-7853724f04a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-9f861559-c2f4-46e9-a6f9-7853724f04a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:36.355766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:36.357362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:36.358701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 77/152] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:36.367927 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681cb75-a5e3-4d21-a5c3-e06bbf7a14a5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:36.506503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058ea57-f4d7-42a6-8386-b1eaf8ea2e2a None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:36.506777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058ea57-f4d7-42a6-8386-b1eaf8ea2e2a None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:36.507116 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058ea57-f4d7-42a6-8386-b1eaf8ea2e2a None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:36.508607 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 78/153] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:34:36.518022 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f062aa2-1515-42a6-b01f-c4d16307b945 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:36.518295 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f062aa2-1515-42a6-b01f-c4d16307b945 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:36.518595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f062aa2-1515-42a6-b01f-c4d16307b945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:36.552156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-9f062aa2-1515-42a6-b01f-c4d16307b945 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:36.553295 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-9f062aa2-1515-42a6-b01f-c4d16307b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:36.624257 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681cb75-a5e3-4d21-a5c3-e06bbf7a14a5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:36.630535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:36.687936 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b9870-d8eb-4fd2-af9d-51ae2956d645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:36.688575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b9870-d8eb-4fd2-af9d-51ae2956d645 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:36.688854 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b9870-d8eb-4fd2-af9d-51ae2956d645 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:36.689122 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b9870-d8eb-4fd2-af9d-51ae2956d645 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:36.689813 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b9870-d8eb-4fd2-af9d-51ae2956d645 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:36.690152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b9870-d8eb-4fd2-af9d-51ae2956d645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:36.692088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b9870-d8eb-4fd2-af9d-51ae2956d645 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:36.705626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 76/154] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:36.707062 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406af476-8992-4227-b636-4821450cacb0 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:36.707417 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406af476-8992-4227-b636-4821450cacb0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:36.707701 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406af476-8992-4227-b636-4821450cacb0 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:36.709000 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 77/155] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:36.716237 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad0fc5-730f-4f47-b5d7-fe64a08fd7d7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:36.716610 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad0fc5-730f-4f47-b5d7-fe64a08fd7d7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:36.716841 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad0fc5-730f-4f47-b5d7-fe64a08fd7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:36.739436 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-40ad0fc5-730f-4f47-b5d7-fe64a08fd7d7 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:36.740899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-40ad0fc5-730f-4f47-b5d7-fe64a08fd7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:36.762016 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 79/156] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:34:36.763628 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f062aa2-1515-42a6-b01f-c4d16307b945 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:36.779203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:36.831372 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 78/157] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:36.993170 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f062aa2-1515-42a6-b01f-c4d16307b945 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:37.052944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fa5e6-8853-4a71-a177-13a3030f3ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:37.053609 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fa5e6-8853-4a71-a177-13a3030f3ab7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:37.053883 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fa5e6-8853-4a71-a177-13a3030f3ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:37.054200 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fa5e6-8853-4a71-a177-13a3030f3ab7 None admin] PATH_INFO: `/v3/users/bbc83d86dc05486faa3364a168ae610d` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:37.055449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fa5e6-8853-4a71-a177-13a3030f3ab7 None admin] RBAC: Authorizing `identity:get_user(user_id=bbc83d86dc05486faa3364a168ae610d)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:37.057027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fa5e6-8853-4a71-a177-13a3030f3ab7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:37.059855 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 80/158] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:36 2022] GET /identity/v3/users/bbc83d86dc05486faa3364a168ae610d => generated 289 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:37.064965 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad0fc5-730f-4f47-b5d7-fe64a08fd7d7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:37.220637 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad0fc5-730f-4f47-b5d7-fe64a08fd7d7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:37.279780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce9560-4bdf-4988-b9f5-995ac228a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:37.280500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce9560-4bdf-4988-b9f5-995ac228a060 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:37.280775 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce9560-4bdf-4988-b9f5-995ac228a060 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:37.281101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce9560-4bdf-4988-b9f5-995ac228a060 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:37.282423 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce9560-4bdf-4988-b9f5-995ac228a060 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:37.283926 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce9560-4bdf-4988-b9f5-995ac228a060 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:37.286015 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 79/159] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:37 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:37.292103 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fa5e6-8853-4a71-a177-13a3030f3ab7 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:37.360339 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:37.474102 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fa5e6-8853-4a71-a177-13a3030f3ab7 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:37.521904 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bbd3e-0c1c-4a83-90c4-61523292cd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:37.522627 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bbd3e-0c1c-4a83-90c4-61523292cd60 None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:37.522906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bbd3e-0c1c-4a83-90c4-61523292cd60 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:37.523181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bbd3e-0c1c-4a83-90c4-61523292cd60 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be/users/bbc83d86dc05486faa3364a168ae610d/roles/1b0fe0772e714d1ba454d992ef8fa266` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:37.525260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bbd3e-0c1c-4a83-90c4-61523292cd60 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f0bb97051994c89b3544850957b48be, user_id=bbc83d86dc05486faa3364a168ae610d, role_id=1b0fe0772e714d1ba454d992ef8fa266)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:37.526924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bbd3e-0c1c-4a83-90c4-61523292cd60 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:37.563173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 81/160] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:34:37 2022] PUT /identity/v3/projects/3f0bb97051994c89b3544850957b48be/users/bbc83d86dc05486faa3364a168ae610d/roles/1b0fe0772e714d1ba454d992ef8fa266 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:34:37.685378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce9560-4bdf-4988-b9f5-995ac228a060 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:37.712654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c0249-bd21-4ddc-8db5-dc69c61f76bd None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:37.712962 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c0249-bd21-4ddc-8db5-dc69c61f76bd None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:37.713210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c0249-bd21-4ddc-8db5-dc69c61f76bd None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:37.714826 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 82/161] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:37.731939 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b6582-d26e-4cdc-9454-b18febe7fa7b None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:37.732138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b6582-d26e-4cdc-9454-b18febe7fa7b None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:37.732402 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b6582-d26e-4cdc-9454-b18febe7fa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:37.771201 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-121b6582-d26e-4cdc-9454-b18febe7fa7b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:37.772455 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-121b6582-d26e-4cdc-9454-b18febe7fa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:37.774908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:37.900651 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 83/162] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:34:37.903192 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b6582-d26e-4cdc-9454-b18febe7fa7b None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:37.922139 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce9560-4bdf-4988-b9f5-995ac228a060 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:37.975683 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:37.998645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7e2db-7c37-4f66-9605-078f21e74667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:37.999655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7e2db-7c37-4f66-9605-078f21e74667 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:38.000225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7e2db-7c37-4f66-9605-078f21e74667 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:38.000705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7e2db-7c37-4f66-9605-078f21e74667 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:38.021823 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7e2db-7c37-4f66-9605-078f21e74667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-05c7e2db-7c37-4f66-9605-078f21e74667 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:38.027829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:38.028742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:38.029588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:38.029588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:38.029588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:38.029588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:38.029588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:38.029588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:38.029588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:38.029588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:38.029588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:38.029588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:38.029588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:34:38.029588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:38.031993 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 80/163] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:34:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:38.038908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7e2db-7c37-4f66-9605-078f21e74667 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:38.180056 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b6582-d26e-4cdc-9454-b18febe7fa7b None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:38.269092 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:38.271044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6c080-71b8-4b94-a106-137d0378babf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:38.272089 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6c080-71b8-4b94-a106-137d0378babf None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:38.273106 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6c080-71b8-4b94-a106-137d0378babf None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:38.273106 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6c080-71b8-4b94-a106-137d0378babf None admin] PATH_INFO: `/v3/roles/member` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:38.281764 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7e2db-7c37-4f66-9605-078f21e74667 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:38.301120 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6c080-71b8-4b94-a106-137d0378babf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-b5f6c080-71b8-4b94-a106-137d0378babf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:38.306598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:38.307478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:38.308235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:38.308235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:38.308235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:38.308235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:38.308235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:38.308235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:38.308235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:38.308235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:38.308235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:38.308235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:38.308235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:38.308235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:38.310727 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 84/164] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:38.317109 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6c080-71b8-4b94-a106-137d0378babf None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:38.369295 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0918663-2e39-46d2-bab6-8e7f60417aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:38.370984 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918663-2e39-46d2-bab6-8e7f60417aed None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:38.372216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918663-2e39-46d2-bab6-8e7f60417aed None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:38.372637 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918663-2e39-46d2-bab6-8e7f60417aed None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:38.377894 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0918663-2e39-46d2-bab6-8e7f60417aed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:38.381259 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0918663-2e39-46d2-bab6-8e7f60417aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:38.382959 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0918663-2e39-46d2-bab6-8e7f60417aed None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:38.408394 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 81/165] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:34:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:38.410248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0918663-2e39-46d2-bab6-8e7f60417aed None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:38.564581 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:38.606154 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6c080-71b8-4b94-a106-137d0378babf None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:38.668649 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0918663-2e39-46d2-bab6-8e7f60417aed None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:38.675739 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178a0aa-57aa-4e91-b423-2e8217a75299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:38.676384 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178a0aa-57aa-4e91-b423-2e8217a75299 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:38.676658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178a0aa-57aa-4e91-b423-2e8217a75299 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:38.676926 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178a0aa-57aa-4e91-b423-2e8217a75299 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:38.701440 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178a0aa-57aa-4e91-b423-2e8217a75299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-5178a0aa-57aa-4e91-b423-2e8217a75299 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:38.710592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:38.711551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:38.712331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:38.712331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:38.712331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:38.712331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:38.712331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:38.712331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:38.712331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:38.712331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:38.712331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:38.712331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:38.712331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:38.712331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:38.713452 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 85/166] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:38.718063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178a0aa-57aa-4e91-b423-2e8217a75299 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:38.737078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff11b72-06ca-4022-adfe-4ceec5c7fe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:38.737743 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff11b72-06ca-4022-adfe-4ceec5c7fe06 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:38.737984 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff11b72-06ca-4022-adfe-4ceec5c7fe06 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:38.738297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff11b72-06ca-4022-adfe-4ceec5c7fe06 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:38.738980 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff11b72-06ca-4022-adfe-4ceec5c7fe06 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:38.739287 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff11b72-06ca-4022-adfe-4ceec5c7fe06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:38.740963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff11b72-06ca-4022-adfe-4ceec5c7fe06 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:38.752138 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 82/167] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:38.753558 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459d765-be96-440c-9520-b13614d6b9a0 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:38.753850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459d765-be96-440c-9520-b13614d6b9a0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:38.754214 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459d765-be96-440c-9520-b13614d6b9a0 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:38.755744 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 83/168] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:38.757243 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ca7e5-0e7d-4a19-9c72-3f5c3f58ea15 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:38.757538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ca7e5-0e7d-4a19-9c72-3f5c3f58ea15 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:38.757967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ca7e5-0e7d-4a19-9c72-3f5c3f58ea15 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:38.759555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 84/169] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:38.760316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ca7e5-0e7d-4a19-9c72-3f5c3f58ea15 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:38.788831 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:38.843964 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178a0aa-57aa-4e91-b423-2e8217a75299 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:38.885406 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa870e-b7fe-40a1-acb0-a485f7efd452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:38.886222 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa870e-b7fe-40a1-acb0-a485f7efd452 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:38.887060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa870e-b7fe-40a1-acb0-a485f7efd452 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:38.887345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa870e-b7fe-40a1-acb0-a485f7efd452 None admin] PATH_INFO: `/v3/users/7563b6e0ea29491291eb70a0be54b165` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:38.889411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa870e-b7fe-40a1-acb0-a485f7efd452 None admin] RBAC: Authorizing `identity:get_user(user_id=7563b6e0ea29491291eb70a0be54b165)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:38.890656 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa870e-b7fe-40a1-acb0-a485f7efd452 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:38.892636 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 86/170] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:38 2022] GET /identity/v3/users/7563b6e0ea29491291eb70a0be54b165 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:38.893843 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa870e-b7fe-40a1-acb0-a485f7efd452 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:38.896647 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ca7e5-0e7d-4a19-9c72-3f5c3f58ea15 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:38.919395 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:38.948475 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9649951-60bc-4418-bf0c-4447f26daf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:38.949990 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9649951-60bc-4418-bf0c-4447f26daf84 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:38.950452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9649951-60bc-4418-bf0c-4447f26daf84 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:38.950889 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9649951-60bc-4418-bf0c-4447f26daf84 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:38.951754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9649951-60bc-4418-bf0c-4447f26daf84 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:38.952201 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9649951-60bc-4418-bf0c-4447f26daf84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:38.953935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9649951-60bc-4418-bf0c-4447f26daf84 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:38.966119 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 85/171] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:38.967906 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36229b6b-3e05-4fff-87c6-1f336e6769e7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:38.968402 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36229b6b-3e05-4fff-87c6-1f336e6769e7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:38.968875 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36229b6b-3e05-4fff-87c6-1f336e6769e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:38.996327 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-36229b6b-3e05-4fff-87c6-1f336e6769e7 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:39.002667 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-36229b6b-3e05-4fff-87c6-1f336e6769e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:39.080853 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa870e-b7fe-40a1-acb0-a485f7efd452 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:39.116433 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 86/172] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:39.118494 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd1ef6-e2da-46ef-8e4a-aedec47f30f1 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:39.118997 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd1ef6-e2da-46ef-8e4a-aedec47f30f1 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:39.119506 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd1ef6-e2da-46ef-8e4a-aedec47f30f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:39.141388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6ebd1ef6-e2da-46ef-8e4a-aedec47f30f1 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:39.143034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6ebd1ef6-e2da-46ef-8e4a-aedec47f30f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:39.149226 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:39.190602 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690fac8-8d05-4d9e-936a-d786e3415977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:39.192893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690fac8-8d05-4d9e-936a-d786e3415977 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:39.194226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690fac8-8d05-4d9e-936a-d786e3415977 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:39.194730 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690fac8-8d05-4d9e-936a-d786e3415977 None admin] PATH_INFO: `/v3/users/7563b6e0ea29491291eb70a0be54b165` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:39.200491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690fac8-8d05-4d9e-936a-d786e3415977 None admin] RBAC: Authorizing `identity:get_user(user_id=7563b6e0ea29491291eb70a0be54b165)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:39.202432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690fac8-8d05-4d9e-936a-d786e3415977 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:39.205170 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 87/173] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:38 2022] GET /identity/v3/users/7563b6e0ea29491291eb70a0be54b165 => generated 275 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:39.207862 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690fac8-8d05-4d9e-936a-d786e3415977 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:39.286271 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 87/174] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:39.292547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd1ef6-e2da-46ef-8e4a-aedec47f30f1 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:39.386089 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:39.429368 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690fac8-8d05-4d9e-936a-d786e3415977 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:39.502471 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d30a1-2303-45fd-989a-06afea563bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:39.503510 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d30a1-2303-45fd-989a-06afea563bc7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:39.504744 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d30a1-2303-45fd-989a-06afea563bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:39.505438 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d30a1-2303-45fd-989a-06afea563bc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:39.507572 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd1ef6-e2da-46ef-8e4a-aedec47f30f1 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:39.592009 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a90e3a-8860-489c-81b9-bdda8468d64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:39.593018 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a90e3a-8860-489c-81b9-bdda8468d64a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:39.593496 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a90e3a-8860-489c-81b9-bdda8468d64a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:39.593991 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a90e3a-8860-489c-81b9-bdda8468d64a None admin] PATH_INFO: `/v3/users/7563b6e0ea29491291eb70a0be54b165` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:39.597697 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a90e3a-8860-489c-81b9-bdda8468d64a None admin] RBAC: Authorizing `identity:get_user(user_id=7563b6e0ea29491291eb70a0be54b165)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:39.600167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a90e3a-8860-489c-81b9-bdda8468d64a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:39.605922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 88/175] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:39 2022] GET /identity/v3/users/7563b6e0ea29491291eb70a0be54b165 => generated 275 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:39.607611 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a90e3a-8860-489c-81b9-bdda8468d64a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:39.621833 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:39.653416 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d30a1-2303-45fd-989a-06afea563bc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:39.656053 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d30a1-2303-45fd-989a-06afea563bc7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:39.772284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a90e3a-8860-489c-81b9-bdda8468d64a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:39.796891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:39.834618 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 88/176] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:39.836228 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781916c-5246-43d5-8090-20b1df7b20ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:39.836735 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a301d4-9434-48e2-a131-3eda8bbb357e None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:39.837306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a301d4-9434-48e2-a131-3eda8bbb357e None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:39.837835 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a301d4-9434-48e2-a131-3eda8bbb357e None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:39.838514 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781916c-5246-43d5-8090-20b1df7b20ca None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:39.839224 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781916c-5246-43d5-8090-20b1df7b20ca None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:39.840449 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781916c-5246-43d5-8090-20b1df7b20ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:39.841274 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 89/177] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:39.843211 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a301d4-9434-48e2-a131-3eda8bbb357e None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:39.979248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781916c-5246-43d5-8090-20b1df7b20ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:39.982562 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781916c-5246-43d5-8090-20b1df7b20ca None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:40.001074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:40.023022 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a301d4-9434-48e2-a131-3eda8bbb357e None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:40.081649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c6b8d-1abf-4590-a77a-3ab9e2175d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:40.082539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c6b8d-1abf-4590-a77a-3ab9e2175d50 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:40.082963 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c6b8d-1abf-4590-a77a-3ab9e2175d50 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:40.083361 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c6b8d-1abf-4590-a77a-3ab9e2175d50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:40.098426 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c6b8d-1abf-4590-a77a-3ab9e2175d50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-c10c6b8d-1abf-4590-a77a-3ab9e2175d50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:40.104065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:40.106088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 90/178] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:34:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:40.107360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c6b8d-1abf-4590-a77a-3ab9e2175d50 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:40.158184 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:40.177802 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 89/179] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:40.179005 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781916c-5246-43d5-8090-20b1df7b20ca None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:40.243369 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c6b8d-1abf-4590-a77a-3ab9e2175d50 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:40.309267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc0bda-2963-4972-b794-c3adcf6b4cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:40.310195 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc0bda-2963-4972-b794-c3adcf6b4cea None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:40.310606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc0bda-2963-4972-b794-c3adcf6b4cea None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:40.311018 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc0bda-2963-4972-b794-c3adcf6b4cea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:40.363919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781916c-5246-43d5-8090-20b1df7b20ca None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:40.382598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:40.415541 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc0bda-2963-4972-b794-c3adcf6b4cea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:40.417509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc0bda-2963-4972-b794-c3adcf6b4cea None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:40.430392 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136da9f-b7de-4d4d-9654-480d88249adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:40.431045 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136da9f-b7de-4d4d-9654-480d88249adc None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:40.431323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136da9f-b7de-4d4d-9654-480d88249adc None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:40.431596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136da9f-b7de-4d4d-9654-480d88249adc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:40.449872 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136da9f-b7de-4d4d-9654-480d88249adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-d136da9f-b7de-4d4d-9654-480d88249adc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:40.457014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:40.458025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 90/180] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:34:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:40.459111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136da9f-b7de-4d4d-9654-480d88249adc None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:40.483909 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:40.562138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 91/181] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:40.562138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3478334-2c81-4fce-a8cb-02e99639080b None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:40.562138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3478334-2c81-4fce-a8cb-02e99639080b None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:40.562138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3478334-2c81-4fce-a8cb-02e99639080b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:40.581654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-b3478334-2c81-4fce-a8cb-02e99639080b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:40.582975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-b3478334-2c81-4fce-a8cb-02e99639080b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:40.605293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136da9f-b7de-4d4d-9654-480d88249adc None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:40.665252 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 92/182] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:40.666307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3478334-2c81-4fce-a8cb-02e99639080b None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:40.673746 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3c3bf-e007-44ff-85ea-4b98932f1fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:40.675501 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3c3bf-e007-44ff-85ea-4b98932f1fab None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:40.676271 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3c3bf-e007-44ff-85ea-4b98932f1fab None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:40.677182 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3c3bf-e007-44ff-85ea-4b98932f1fab None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:40.679809 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3c3bf-e007-44ff-85ea-4b98932f1fab None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:40.685070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3c3bf-e007-44ff-85ea-4b98932f1fab None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:40.689022 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 91/183] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:40 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:40.691256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3c3bf-e007-44ff-85ea-4b98932f1fab None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:40.741126 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:40.772711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:40.818673 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3478334-2c81-4fce-a8cb-02e99639080b None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:40.863564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642662e-5a7c-4f6f-8a61-c0601077d757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:40.864726 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642662e-5a7c-4f6f-8a61-c0601077d757 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:40.865233 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642662e-5a7c-4f6f-8a61-c0601077d757 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:40.866161 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642662e-5a7c-4f6f-8a61-c0601077d757 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:40.867084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642662e-5a7c-4f6f-8a61-c0601077d757 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:40.867917 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642662e-5a7c-4f6f-8a61-c0601077d757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:40.869533 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642662e-5a7c-4f6f-8a61-c0601077d757 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:40.882765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 93/184] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:34:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:40.886444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3c3bf-e007-44ff-85ea-4b98932f1fab None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:40.888042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642662e-5a7c-4f6f-8a61-c0601077d757 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:40.947310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7455a-0210-4d8d-a5c6-da720e79d593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:40.947953 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7455a-0210-4d8d-a5c6-da720e79d593 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:40.948234 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7455a-0210-4d8d-a5c6-da720e79d593 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:40.948513 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7455a-0210-4d8d-a5c6-da720e79d593 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:40.949697 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7455a-0210-4d8d-a5c6-da720e79d593 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:40.951200 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7455a-0210-4d8d-a5c6-da720e79d593 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:40.953113 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 92/185] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:40 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:40.953634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7455a-0210-4d8d-a5c6-da720e79d593 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:41.007787 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642662e-5a7c-4f6f-8a61-c0601077d757 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:41.038647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:41.045606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299c12f-eb87-4d97-b88a-9a18df08b9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:41.046379 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c12f-eb87-4d97-b88a-9a18df08b9ef None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:41.046789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c12f-eb87-4d97-b88a-9a18df08b9ef None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:41.047160 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299c12f-eb87-4d97-b88a-9a18df08b9ef None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:41.047904 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299c12f-eb87-4d97-b88a-9a18df08b9ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:41.048223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299c12f-eb87-4d97-b88a-9a18df08b9ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:41.049768 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299c12f-eb87-4d97-b88a-9a18df08b9ef None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:41.059895 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 94/186] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:41.060495 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299c12f-eb87-4d97-b88a-9a18df08b9ef None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:41.067346 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7455a-0210-4d8d-a5c6-da720e79d593 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:41.070636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:41.106525 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba71ba-0430-4581-88a2-67361becb542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:41.107165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba71ba-0430-4581-88a2-67361becb542 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:41.107443 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba71ba-0430-4581-88a2-67361becb542 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:41.107711 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba71ba-0430-4581-88a2-67361becb542 None admin] PATH_INFO: `/v3/projects/48446be76b3048ebba9328ab935f4256` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:41.108889 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba71ba-0430-4581-88a2-67361becb542 None admin] RBAC: Authorizing `identity:get_project(project_id=48446be76b3048ebba9328ab935f4256)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:41.110410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba71ba-0430-4581-88a2-67361becb542 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:41.112433 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 93/187] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:40 2022] GET /identity/v3/projects/48446be76b3048ebba9328ab935f4256 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:41.113238 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba71ba-0430-4581-88a2-67361becb542 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:41.179237 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299c12f-eb87-4d97-b88a-9a18df08b9ef None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:41.219450 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4ba45-485e-4b48-82ae-df3fdd94284f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:41.220414 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4ba45-485e-4b48-82ae-df3fdd94284f None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:41.220837 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4ba45-485e-4b48-82ae-df3fdd94284f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:41.221249 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4ba45-485e-4b48-82ae-df3fdd94284f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:41.222185 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4ba45-485e-4b48-82ae-df3fdd94284f None admin] RBAC: Adding query filter params (role.id=ca13aabb28b6442f8540fa4e8bb1b35a, scope.project.id=3f0bb97051994c89b3544850957b48be, user.id=7563b6e0ea29491291eb70a0be54b165) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:41.222599 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4ba45-485e-4b48-82ae-df3fdd94284f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:41.224198 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4ba45-485e-4b48-82ae-df3fdd94284f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:41.231529 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 95/188] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:41 2022] GET /identity/v3/role_assignments?user.id=7563b6e0ea29491291eb70a0be54b165&scope.project.id=3f0bb97051994c89b3544850957b48be&role.id=ca13aabb28b6442f8540fa4e8bb1b35a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:41.232411 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba71ba-0430-4581-88a2-67361becb542 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:41.232836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b85f79-3408-417b-8953-a11516fdf2df None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:41.233122 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b85f79-3408-417b-8953-a11516fdf2df None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:41.233395 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b85f79-3408-417b-8953-a11516fdf2df None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:41.235084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 96/189] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:41.235612 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b85f79-3408-417b-8953-a11516fdf2df None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:41.299328 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9cdef-a8c4-4ded-aa31-80dedda2aad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:41.300364 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9cdef-a8c4-4ded-aa31-80dedda2aad0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:41.300799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9cdef-a8c4-4ded-aa31-80dedda2aad0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:41.301203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9cdef-a8c4-4ded-aa31-80dedda2aad0 None admin] PATH_INFO: `/v3/users/56bf68b0201e419781232cd507f5b7cb` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:41.311936 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:41.322377 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9cdef-a8c4-4ded-aa31-80dedda2aad0 None admin] RBAC: Authorizing `identity:get_user(user_id=56bf68b0201e419781232cd507f5b7cb)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:41.323879 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9cdef-a8c4-4ded-aa31-80dedda2aad0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:41.326502 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 94/190] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:41 2022] GET /identity/v3/users/56bf68b0201e419781232cd507f5b7cb => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:41.326924 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9cdef-a8c4-4ded-aa31-80dedda2aad0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:41.367275 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:41.386014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b85f79-3408-417b-8953-a11516fdf2df None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:41.459050 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3518878-ad9c-4fe6-9a44-31b073a801fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:41.459710 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3518878-ad9c-4fe6-9a44-31b073a801fa None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:41.459990 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3518878-ad9c-4fe6-9a44-31b073a801fa None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:41.460264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3518878-ad9c-4fe6-9a44-31b073a801fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:41.462293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3518878-ad9c-4fe6-9a44-31b073a801fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:41.463154 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3518878-ad9c-4fe6-9a44-31b073a801fa None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:41.466401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 97/191] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:41.471360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3518878-ad9c-4fe6-9a44-31b073a801fa None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:41.471609 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9cdef-a8c4-4ded-aa31-80dedda2aad0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:41.535093 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b2e89-f8df-4ad2-80a7-bfe1f4abdfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:41.535754 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b2e89-f8df-4ad2-80a7-bfe1f4abdfd5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:41.536040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b2e89-f8df-4ad2-80a7-bfe1f4abdfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:41.536287 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b2e89-f8df-4ad2-80a7-bfe1f4abdfd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:41.537040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b2e89-f8df-4ad2-80a7-bfe1f4abdfd5 None admin] RBAC: Adding query filter params (role.id=d18e4897bf3648498a472a2ea8174f8b, scope.project.id=3f0bb97051994c89b3544850957b48be, user.id=7563b6e0ea29491291eb70a0be54b165) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:41.537350 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b2e89-f8df-4ad2-80a7-bfe1f4abdfd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:41.538863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b2e89-f8df-4ad2-80a7-bfe1f4abdfd5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:41.547373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 95/192] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:41 2022] GET /identity/v3/role_assignments?user.id=7563b6e0ea29491291eb70a0be54b165&scope.project.id=3f0bb97051994c89b3544850957b48be&role.id=d18e4897bf3648498a472a2ea8174f8b => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:41.548630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b2e89-f8df-4ad2-80a7-bfe1f4abdfd5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:41.608510 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3518878-ad9c-4fe6-9a44-31b073a801fa None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:41.640327 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:41.654808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1feea-652f-4faf-a36c-71b75d6d5212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:41.655895 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1feea-652f-4faf-a36c-71b75d6d5212 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:41.656260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1feea-652f-4faf-a36c-71b75d6d5212 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:41.656699 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1feea-652f-4faf-a36c-71b75d6d5212 None admin] PATH_INFO: `/v3/users/56bf68b0201e419781232cd507f5b7cb` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:41.658794 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1feea-652f-4faf-a36c-71b75d6d5212 None admin] RBAC: Authorizing `identity:get_user(user_id=56bf68b0201e419781232cd507f5b7cb)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:41.660386 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1feea-652f-4faf-a36c-71b75d6d5212 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:41.662331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 98/193] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:41 2022] GET /identity/v3/users/56bf68b0201e419781232cd507f5b7cb => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:41.663482 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8082629-ba00-41da-8af9-289dffb808aa None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:41.663892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8082629-ba00-41da-8af9-289dffb808aa None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:41.664170 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8082629-ba00-41da-8af9-289dffb808aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:41.696938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-f8082629-ba00-41da-8af9-289dffb808aa None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:41.698415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-f8082629-ba00-41da-8af9-289dffb808aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:41.729908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:41.744238 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b2e89-f8df-4ad2-80a7-bfe1f4abdfd5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:41.796508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 99/194] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:41.798374 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8082629-ba00-41da-8af9-289dffb808aa None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:41.817140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c16b3-86e0-4983-8f82-716fbb75681c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:41.818219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c16b3-86e0-4983-8f82-716fbb75681c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:41.818686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c16b3-86e0-4983-8f82-716fbb75681c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:41.819162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c16b3-86e0-4983-8f82-716fbb75681c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:41.820892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c16b3-86e0-4983-8f82-716fbb75681c None admin] RBAC: Adding query filter params (role.id=d18e4897bf3648498a472a2ea8174f8b, scope.project.id=48446be76b3048ebba9328ab935f4256, user.id=7563b6e0ea29491291eb70a0be54b165) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:41.821424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c16b3-86e0-4983-8f82-716fbb75681c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:41.825229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c16b3-86e0-4983-8f82-716fbb75681c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:41.837499 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 96/195] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:41 2022] GET /identity/v3/role_assignments?user.id=7563b6e0ea29491291eb70a0be54b165&scope.project.id=48446be76b3048ebba9328ab935f4256&role.id=d18e4897bf3648498a472a2ea8174f8b => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:41.840306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c16b3-86e0-4983-8f82-716fbb75681c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:41.939192 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8082629-ba00-41da-8af9-289dffb808aa None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:41.982693 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:42.001764 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a7a9d-8a79-4081-8ad1-678175b83589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:42.002449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a7a9d-8a79-4081-8ad1-678175b83589 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:42.003344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a7a9d-8a79-4081-8ad1-678175b83589 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:42.003644 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a7a9d-8a79-4081-8ad1-678175b83589 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:42.036173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c16b3-86e0-4983-8f82-716fbb75681c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:42.078651 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752ab44-8e48-4898-be39-62b54b0bb518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:42.079279 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752ab44-8e48-4898-be39-62b54b0bb518 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:42.079554 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752ab44-8e48-4898-be39-62b54b0bb518 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:42.079825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752ab44-8e48-4898-be39-62b54b0bb518 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:42.080498 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752ab44-8e48-4898-be39-62b54b0bb518 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:42.082140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752ab44-8e48-4898-be39-62b54b0bb518 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:42.103767 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a7a9d-8a79-4081-8ad1-678175b83589 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:42.105597 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4a7a9d-8a79-4081-8ad1-678175b83589 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:42.109149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 97/196] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:34:41 2022] POST /identity/v3/users => generated 297 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:34:42.110085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752ab44-8e48-4898-be39-62b54b0bb518 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:42.131718 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:42.278019 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 100/197] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:42.279116 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752ab44-8e48-4898-be39-62b54b0bb518 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:42.279567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a7a9d-8a79-4081-8ad1-678175b83589 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:42.332299 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f2d7a-8cb0-4992-b789-6943144122e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:42.333128 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f2d7a-8cb0-4992-b789-6943144122e2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:42.333406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f2d7a-8cb0-4992-b789-6943144122e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:42.333903 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:42.335433 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f2d7a-8cb0-4992-b789-6943144122e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:42.427724 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f2d7a-8cb0-4992-b789-6943144122e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:42.430859 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f2d7a-8cb0-4992-b789-6943144122e2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:42.447641 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a7a9d-8a79-4081-8ad1-678175b83589 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:42.457700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:42.508905 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c9838-1472-4b16-b850-aa30ac3ba48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:42.508905 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c9838-1472-4b16-b850-aa30ac3ba48b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:42.509176 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c9838-1472-4b16-b850-aa30ac3ba48b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:42.509431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c9838-1472-4b16-b850-aa30ac3ba48b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:42.527060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c9838-1472-4b16-b850-aa30ac3ba48b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-085c9838-1472-4b16-b850-aa30ac3ba48b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:42.533427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:42.534430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:42.535206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 101/198] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:42.535863 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c9838-1472-4b16-b850-aa30ac3ba48b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:42.600894 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 98/199] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:42.602130 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f2d7a-8cb0-4992-b789-6943144122e2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:42.654788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:42.680227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c9838-1472-4b16-b850-aa30ac3ba48b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:42.724236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e617404-0dbf-47c0-b272-fcb4d8408aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:42.725874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e617404-0dbf-47c0-b272-fcb4d8408aed None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:42.726349 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e617404-0dbf-47c0-b272-fcb4d8408aed None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:42.726944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e617404-0dbf-47c0-b272-fcb4d8408aed None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:42.729217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e617404-0dbf-47c0-b272-fcb4d8408aed None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:42.732461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e617404-0dbf-47c0-b272-fcb4d8408aed None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:42.734901 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 102/200] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:42 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:42.735560 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f2d7a-8cb0-4992-b789-6943144122e2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:42.736181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e617404-0dbf-47c0-b272-fcb4d8408aed None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:42.754186 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:42.785270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4ba0a-f41a-4624-999a-accd5000cac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:42.785810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4ba0a-f41a-4624-999a-accd5000cac9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:42.786113 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4ba0a-f41a-4624-999a-accd5000cac9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:42.786390 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4ba0a-f41a-4624-999a-accd5000cac9 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:42.787067 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4ba0a-f41a-4624-999a-accd5000cac9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:42.787659 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4ba0a-f41a-4624-999a-accd5000cac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:42.790079 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4ba0a-f41a-4624-999a-accd5000cac9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:42.802326 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 99/201] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:42.803568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4ba0a-f41a-4624-999a-accd5000cac9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:42.878731 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e617404-0dbf-47c0-b272-fcb4d8408aed None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:42.939241 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4ba0a-f41a-4624-999a-accd5000cac9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:42.941508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1cd09-e41c-4c29-b8f4-5d3ab0637b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:42.943264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1cd09-e41c-4c29-b8f4-5d3ab0637b88 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:42.943569 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1cd09-e41c-4c29-b8f4-5d3ab0637b88 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:42.943848 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1cd09-e41c-4c29-b8f4-5d3ab0637b88 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:42.945305 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1cd09-e41c-4c29-b8f4-5d3ab0637b88 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:42.946924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1cd09-e41c-4c29-b8f4-5d3ab0637b88 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:42.949716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 103/202] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:42 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:42.951229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1cd09-e41c-4c29-b8f4-5d3ab0637b88 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:42.982058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:42.983858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fc7ef-8c7d-4421-85c3-dcc2cd5de0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:42.984507 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fc7ef-8c7d-4421-85c3-dcc2cd5de0c3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:42.984773 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fc7ef-8c7d-4421-85c3-dcc2cd5de0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:42.985050 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fc7ef-8c7d-4421-85c3-dcc2cd5de0c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:42.985884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fc7ef-8c7d-4421-85c3-dcc2cd5de0c3 None admin] RBAC: Adding query filter params (role.id=ca13aabb28b6442f8540fa4e8bb1b35a, scope.project.id=8f97028a6a564e6b924279df149adb74, user.id=56bf68b0201e419781232cd507f5b7cb) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:42.986237 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fc7ef-8c7d-4421-85c3-dcc2cd5de0c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:42.987937 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fc7ef-8c7d-4421-85c3-dcc2cd5de0c3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:42.995388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 100/203] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:42 2022] GET /identity/v3/role_assignments?user.id=56bf68b0201e419781232cd507f5b7cb&scope.project.id=8f97028a6a564e6b924279df149adb74&role.id=ca13aabb28b6442f8540fa4e8bb1b35a => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:42.996382 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fc7ef-8c7d-4421-85c3-dcc2cd5de0c3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:43.121707 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:43.137849 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1cd09-e41c-4c29-b8f4-5d3ab0637b88 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:43.232462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8acf1-116e-4dd7-a780-3abf48fac4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:43.233118 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8acf1-116e-4dd7-a780-3abf48fac4c1 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:43.233403 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8acf1-116e-4dd7-a780-3abf48fac4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:43.233678 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8acf1-116e-4dd7-a780-3abf48fac4c1 None admin] PATH_INFO: `/v3/users/bbc83d86dc05486faa3364a168ae610d` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:43.234922 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8acf1-116e-4dd7-a780-3abf48fac4c1 None admin] RBAC: Authorizing `identity:get_user(user_id=bbc83d86dc05486faa3364a168ae610d)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:43.236536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8acf1-116e-4dd7-a780-3abf48fac4c1 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:43.242604 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 104/204] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:42 2022] GET /identity/v3/users/bbc83d86dc05486faa3364a168ae610d => generated 289 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:43.244426 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8acf1-116e-4dd7-a780-3abf48fac4c1 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:43.247705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fc7ef-8c7d-4421-85c3-dcc2cd5de0c3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:43.316058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba52a72-db7e-4554-8191-cb74e10f38cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:43.317043 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba52a72-db7e-4554-8191-cb74e10f38cc None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:43.317538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba52a72-db7e-4554-8191-cb74e10f38cc None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:43.318004 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba52a72-db7e-4554-8191-cb74e10f38cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:43.319082 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba52a72-db7e-4554-8191-cb74e10f38cc None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=8f97028a6a564e6b924279df149adb74, user.id=56bf68b0201e419781232cd507f5b7cb) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:43.319608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba52a72-db7e-4554-8191-cb74e10f38cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:43.321268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba52a72-db7e-4554-8191-cb74e10f38cc None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:43.328616 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 101/205] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:42 2022] GET /identity/v3/role_assignments?user.id=56bf68b0201e419781232cd507f5b7cb&scope.project.id=8f97028a6a564e6b924279df149adb74&role.id=fc68d072459d4720b9666c93bd535d63 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:43.426179 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:43.455811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8acf1-116e-4dd7-a780-3abf48fac4c1 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:43.527227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294e499-f57a-4de7-8299-118e70751c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:43.528212 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294e499-f57a-4de7-8299-118e70751c0b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:43.528681 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294e499-f57a-4de7-8299-118e70751c0b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:43.529142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294e499-f57a-4de7-8299-118e70751c0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:43.651909 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294e499-f57a-4de7-8299-118e70751c0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:43.654633 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294e499-f57a-4de7-8299-118e70751c0b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:43.833179 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:43.857707 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 105/206] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 12:34:43.866129 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba52a72-db7e-4554-8191-cb74e10f38cc None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:44.047009 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba52a72-db7e-4554-8191-cb74e10f38cc None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:44.049689 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:44.098870 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40392af7-3a53-4a4b-bee1-1e173819cc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:44.099407 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40392af7-3a53-4a4b-bee1-1e173819cc84 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:44.099681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40392af7-3a53-4a4b-bee1-1e173819cc84 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:44.099950 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40392af7-3a53-4a4b-bee1-1e173819cc84 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:44.106879 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40392af7-3a53-4a4b-bee1-1e173819cc84 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:44.108365 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40392af7-3a53-4a4b-bee1-1e173819cc84 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:44.110337 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 102/207] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:43 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:44.120218 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294e499-f57a-4de7-8299-118e70751c0b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:44.375223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294e499-f57a-4de7-8299-118e70751c0b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:44.452967 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13321d-da27-44b3-93d1-778fbe9ab1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:44.453602 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13321d-da27-44b3-93d1-778fbe9ab1ba None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:44.453885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13321d-da27-44b3-93d1-778fbe9ab1ba None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:44.454238 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13321d-da27-44b3-93d1-778fbe9ab1ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:44.454993 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13321d-da27-44b3-93d1-778fbe9ab1ba None admin] RBAC: Adding query filter params (role.id=1b0fe0772e714d1ba454d992ef8fa266, scope.project.id=3f0bb97051994c89b3544850957b48be, user.id=bbc83d86dc05486faa3364a168ae610d) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:44.455302 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13321d-da27-44b3-93d1-778fbe9ab1ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:44.456816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13321d-da27-44b3-93d1-778fbe9ab1ba None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:44.464034 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 106/208] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:44 2022] GET /identity/v3/role_assignments?user.id=bbc83d86dc05486faa3364a168ae610d&scope.project.id=3f0bb97051994c89b3544850957b48be&role.id=1b0fe0772e714d1ba454d992ef8fa266 => generated 608 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:45.308905 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6285487-bf57-4629-984f-e961184cc957 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:45.309654 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6285487-bf57-4629-984f-e961184cc957 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:45.309654 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6285487-bf57-4629-984f-e961184cc957 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:45.310859 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 103/209] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:45.320056 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908c29a-db34-4a28-995f-bf8e12bcc268 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:45.320573 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908c29a-db34-4a28-995f-bf8e12bcc268 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:45.321016 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908c29a-db34-4a28-995f-bf8e12bcc268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:45.357200 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-f908c29a-db34-4a28-995f-bf8e12bcc268 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:45.359390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-f908c29a-db34-4a28-995f-bf8e12bcc268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:45.454083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:45.492362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 107/210] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:45.796033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269dd182-7d53-4647-9d4d-dacfeaf4708a None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:45.796606 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269dd182-7d53-4647-9d4d-dacfeaf4708a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:45.797105 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269dd182-7d53-4647-9d4d-dacfeaf4708a None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:45.798998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 104/211] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:45.808941 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149d968-41c3-401b-b152-ee9d4cbe1b37 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:45.809382 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149d968-41c3-401b-b152-ee9d4cbe1b37 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:45.809817 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149d968-41c3-401b-b152-ee9d4cbe1b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:45.846638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-5149d968-41c3-401b-b152-ee9d4cbe1b37 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:45.848414 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-5149d968-41c3-401b-b152-ee9d4cbe1b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:45.934329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 108/212] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:45.982900 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269dd182-7d53-4647-9d4d-dacfeaf4708a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:46.188660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269dd182-7d53-4647-9d4d-dacfeaf4708a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:46.243954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d38f3b-c200-470b-9799-4edd36f3a062 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:46.244596 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d38f3b-c200-470b-9799-4edd36f3a062 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:46.245074 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d38f3b-c200-470b-9799-4edd36f3a062 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:46.247604 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 109/213] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:46.253888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d040b62-508a-4aa7-8a7a-4dd498dc4600 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:46.254273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df322eb4-ce76-4a87-8388-996ac2694b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:46.254767 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d040b62-508a-4aa7-8a7a-4dd498dc4600 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:46.255262 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d040b62-508a-4aa7-8a7a-4dd498dc4600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:46.255365 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df322eb4-ce76-4a87-8388-996ac2694b46 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:46.255799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df322eb4-ce76-4a87-8388-996ac2694b46 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:46.256272 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df322eb4-ce76-4a87-8388-996ac2694b46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:46.277099 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df322eb4-ce76-4a87-8388-996ac2694b46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:46.277991 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-9d040b62-508a-4aa7-8a7a-4dd498dc4600 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:46.279454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-9d040b62-508a-4aa7-8a7a-4dd498dc4600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-df322eb4-ce76-4a87-8388-996ac2694b46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:46.287069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:46.288042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:46.288827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:46.288827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:46.288827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:46.288827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:46.288827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:46.288827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:46.288827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:46.288827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:46.288827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:46.288827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:46.288827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:46.288827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:46.289966 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 105/214] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:46.297267 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df322eb4-ce76-4a87-8388-996ac2694b46 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:46.373369 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 110/215] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:46.417782 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:46.468597 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d040b62-508a-4aa7-8a7a-4dd498dc4600 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:46.506981 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df322eb4-ce76-4a87-8388-996ac2694b46 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:46.562504 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:46.590656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfaa5b9-2de6-4412-aaeb-27ae006616ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:46.591866 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfaa5b9-2de6-4412-aaeb-27ae006616ab None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:46.592636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfaa5b9-2de6-4412-aaeb-27ae006616ab None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:46.593518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfaa5b9-2de6-4412-aaeb-27ae006616ab None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:46.595017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfaa5b9-2de6-4412-aaeb-27ae006616ab None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:46.595761 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfaa5b9-2de6-4412-aaeb-27ae006616ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:46.599085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfaa5b9-2de6-4412-aaeb-27ae006616ab None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:46.619729 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 106/216] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:46.629127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfaa5b9-2de6-4412-aaeb-27ae006616ab None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:46.675300 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d040b62-508a-4aa7-8a7a-4dd498dc4600 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:46.716078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91766420-ab0c-4454-9d4c-5a916837465e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:46.716928 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91766420-ab0c-4454-9d4c-5a916837465e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:46.717238 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91766420-ab0c-4454-9d4c-5a916837465e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:46.717487 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91766420-ab0c-4454-9d4c-5a916837465e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:46.718996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91766420-ab0c-4454-9d4c-5a916837465e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:46.720562 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91766420-ab0c-4454-9d4c-5a916837465e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:46.724458 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 111/217] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:46.724458 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab23ee-ff80-46b7-8f47-121612af07c0 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:46.724458 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab23ee-ff80-46b7-8f47-121612af07c0 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:46.724943 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab23ee-ff80-46b7-8f47-121612af07c0 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:46.726717 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 112/218] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:46.727529 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab23ee-ff80-46b7-8f47-121612af07c0 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:46.753452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:46.763316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfaa5b9-2de6-4412-aaeb-27ae006616ab None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:46.814925 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8999cb-aab2-468c-be49-e7ba3479267d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:46.815819 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8999cb-aab2-468c-be49-e7ba3479267d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:46.816268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8999cb-aab2-468c-be49-e7ba3479267d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:46.816712 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8999cb-aab2-468c-be49-e7ba3479267d None admin] PATH_INFO: `/v3/users/25ee7eabd1d04f4381194934cb91b150` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:46.834563 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8999cb-aab2-468c-be49-e7ba3479267d None admin] RBAC: Authorizing `identity:get_user(user_id=25ee7eabd1d04f4381194934cb91b150)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:46.836221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8999cb-aab2-468c-be49-e7ba3479267d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:46.838160 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 107/219] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:46 2022] GET /identity/v3/users/25ee7eabd1d04f4381194934cb91b150 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:46.839526 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8999cb-aab2-468c-be49-e7ba3479267d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:46.875212 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab23ee-ff80-46b7-8f47-121612af07c0 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:46.888041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:46.924571 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9158c5a-7d4c-4918-a978-59965a2b75aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:46.925451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9158c5a-7d4c-4918-a978-59965a2b75aa None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:46.925816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9158c5a-7d4c-4918-a978-59965a2b75aa None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:46.926223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9158c5a-7d4c-4918-a978-59965a2b75aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:46.940628 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9158c5a-7d4c-4918-a978-59965a2b75aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-a9158c5a-7d4c-4918-a978-59965a2b75aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:46.946491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:46.948220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:46.950068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 113/220] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:34:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:46.953031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fffae-382f-4b54-badf-d43a3d40939d None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:46.953031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fffae-382f-4b54-badf-d43a3d40939d None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:46.953031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fffae-382f-4b54-badf-d43a3d40939d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:46.976284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-1d7fffae-382f-4b54-badf-d43a3d40939d None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:46.977747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-1d7fffae-382f-4b54-badf-d43a3d40939d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:46.980571 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8999cb-aab2-468c-be49-e7ba3479267d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:47.026328 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99686a7a-249a-4c38-9df7-6b8e8d47c4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:47.027865 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99686a7a-249a-4c38-9df7-6b8e8d47c4bf None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:47.028322 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99686a7a-249a-4c38-9df7-6b8e8d47c4bf None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:47.028759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99686a7a-249a-4c38-9df7-6b8e8d47c4bf None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:47.029905 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99686a7a-249a-4c38-9df7-6b8e8d47c4bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:47.032455 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99686a7a-249a-4c38-9df7-6b8e8d47c4bf None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:47.061746 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 108/221] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:34:46 2022] POST /identity/v3/users => generated 297 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:34:47.063332 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99686a7a-249a-4c38-9df7-6b8e8d47c4bf None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:47.082388 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 114/222] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:47.082388 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fffae-382f-4b54-badf-d43a3d40939d None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:47.096011 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:47.276703 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99686a7a-249a-4c38-9df7-6b8e8d47c4bf None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:47.326112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:47.332181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fffae-382f-4b54-badf-d43a3d40939d None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:47.356332 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d99e6-f8f8-465b-939f-91715ce0f347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:47.358934 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d99e6-f8f8-465b-939f-91715ce0f347 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:47.359524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d99e6-f8f8-465b-939f-91715ce0f347 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:47.360081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d99e6-f8f8-465b-939f-91715ce0f347 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:47.409758 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fff579-95b1-4bd4-b2c8-acaee96823bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:47.410761 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fff579-95b1-4bd4-b2c8-acaee96823bc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:47.414321 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fff579-95b1-4bd4-b2c8-acaee96823bc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:47.414321 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fff579-95b1-4bd4-b2c8-acaee96823bc None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:47.414321 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fff579-95b1-4bd4-b2c8-acaee96823bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:47.414321 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fff579-95b1-4bd4-b2c8-acaee96823bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:47.415098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fff579-95b1-4bd4-b2c8-acaee96823bc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:47.435590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 115/223] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:47.445614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fff579-95b1-4bd4-b2c8-acaee96823bc None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:47.506426 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d99e6-f8f8-465b-939f-91715ce0f347 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:47.510352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d99e6-f8f8-465b-939f-91715ce0f347 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:47.529254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:47.657235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fff579-95b1-4bd4-b2c8-acaee96823bc None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:47.713039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 109/224] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:47.714547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d99e6-f8f8-465b-939f-91715ce0f347 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:47.724087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7289e-b50c-4fc1-a7c3-6cba49102813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:47.725073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7289e-b50c-4fc1-a7c3-6cba49102813 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:47.725516 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7289e-b50c-4fc1-a7c3-6cba49102813 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:47.725963 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7289e-b50c-4fc1-a7c3-6cba49102813 None admin] PATH_INFO: `/v3/users/56bf68b0201e419781232cd507f5b7cb` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:47.727975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7289e-b50c-4fc1-a7c3-6cba49102813 None admin] RBAC: Authorizing `identity:get_user(user_id=56bf68b0201e419781232cd507f5b7cb)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:47.732245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7289e-b50c-4fc1-a7c3-6cba49102813 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:47.735432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 116/225] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:47 2022] GET /identity/v3/users/56bf68b0201e419781232cd507f5b7cb => generated 283 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:47.736644 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7289e-b50c-4fc1-a7c3-6cba49102813 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:47.796906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:47.911709 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d99e6-f8f8-465b-939f-91715ce0f347 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:47.930859 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:47.945222 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7289e-b50c-4fc1-a7c3-6cba49102813 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:47.988142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9302e3-3cae-4041-9da7-efc799ac1979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:47.989148 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9302e3-3cae-4041-9da7-efc799ac1979 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:47.989609 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9302e3-3cae-4041-9da7-efc799ac1979 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:47.990221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9302e3-3cae-4041-9da7-efc799ac1979 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:48.011000 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9302e3-3cae-4041-9da7-efc799ac1979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:48.012438 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c6651-838a-42b0-a18b-3e3c566cb7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:48.013529 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c6651-838a-42b0-a18b-3e3c566cb7f9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:48.014173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c6651-838a-42b0-a18b-3e3c566cb7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:48.014719 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c6651-838a-42b0-a18b-3e3c566cb7f9 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:48.016818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c6651-838a-42b0-a18b-3e3c566cb7f9 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:48.019545 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c6651-838a-42b0-a18b-3e3c566cb7f9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-8c9302e3-3cae-4041-9da7-efc799ac1979 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:48.022677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:48.024805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:48.026381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:48.026381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:48.026381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:48.026381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:48.026381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:48.026381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:48.026381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:48.026381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:48.026381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:48.026381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:48.026381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:34:48.026381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:48.028896 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 117/226] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:47 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:48.028896 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c6651-838a-42b0-a18b-3e3c566cb7f9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:48.032420 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 110/227] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:34:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:48.034371 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9302e3-3cae-4041-9da7-efc799ac1979 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:48.219151 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:48.223603 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c6651-838a-42b0-a18b-3e3c566cb7f9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:48.232668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9302e3-3cae-4041-9da7-efc799ac1979 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:48.287618 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf9dcc-09fc-459a-8252-c8ad7f2cab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:48.288674 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf9dcc-09fc-459a-8252-c8ad7f2cab89 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:48.289169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf9dcc-09fc-459a-8252-c8ad7f2cab89 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:48.289654 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf9dcc-09fc-459a-8252-c8ad7f2cab89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:48.290669 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf9dcc-09fc-459a-8252-c8ad7f2cab89 None admin] RBAC: Adding query filter params (role.id=d18e4897bf3648498a472a2ea8174f8b, scope.project.id=8f97028a6a564e6b924279df149adb74, user.id=25ee7eabd1d04f4381194934cb91b150) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:48.291198 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf9dcc-09fc-459a-8252-c8ad7f2cab89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:48.292905 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf9dcc-09fc-459a-8252-c8ad7f2cab89 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:48.300222 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 111/228] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:48 2022] GET /identity/v3/role_assignments?user.id=25ee7eabd1d04f4381194934cb91b150&scope.project.id=8f97028a6a564e6b924279df149adb74&role.id=d18e4897bf3648498a472a2ea8174f8b => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:48.301467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf9dcc-09fc-459a-8252-c8ad7f2cab89 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:48.302771 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97069b-5122-4158-aa72-e3feddb89638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:48.303863 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97069b-5122-4158-aa72-e3feddb89638 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:48.304355 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97069b-5122-4158-aa72-e3feddb89638 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:48.304836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97069b-5122-4158-aa72-e3feddb89638 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:48.309169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97069b-5122-4158-aa72-e3feddb89638 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:48.311947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97069b-5122-4158-aa72-e3feddb89638 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:48.315249 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 118/229] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:48 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:48.326474 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97069b-5122-4158-aa72-e3feddb89638 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:48.382013 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:48.572434 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf9dcc-09fc-459a-8252-c8ad7f2cab89 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:48.586227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97069b-5122-4158-aa72-e3feddb89638 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:48.672123 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef35f27-3226-495d-8e1a-83adb470564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:48.673783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef35f27-3226-495d-8e1a-83adb470564a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:48.674617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef35f27-3226-495d-8e1a-83adb470564a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:48.675301 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef35f27-3226-495d-8e1a-83adb470564a None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:48.676724 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef35f27-3226-495d-8e1a-83adb470564a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:48.677452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef35f27-3226-495d-8e1a-83adb470564a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:48.679971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e38242-90ee-485d-9fe5-ef128ed983dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:48.680765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef35f27-3226-495d-8e1a-83adb470564a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:48.680928 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e38242-90ee-485d-9fe5-ef128ed983dd None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:48.681250 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e38242-90ee-485d-9fe5-ef128ed983dd None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:48.681669 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e38242-90ee-485d-9fe5-ef128ed983dd None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74/users/56bf68b0201e419781232cd507f5b7cb/roles/fc68d072459d4720b9666c93bd535d63` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:48.683914 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e38242-90ee-485d-9fe5-ef128ed983dd None admin] RBAC: Authorizing `identity:create_grant(project_id=8f97028a6a564e6b924279df149adb74, user_id=56bf68b0201e419781232cd507f5b7cb, role_id=fc68d072459d4720b9666c93bd535d63)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:48.685513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e38242-90ee-485d-9fe5-ef128ed983dd None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:48.700292 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 112/230] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:34:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:48.709051 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef35f27-3226-495d-8e1a-83adb470564a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:48.742077 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 119/231] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:34:48 2022] PUT /identity/v3/projects/8f97028a6a564e6b924279df149adb74/users/56bf68b0201e419781232cd507f5b7cb/roles/fc68d072459d4720b9666c93bd535d63 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:34:48.925456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:48.940298 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef35f27-3226-495d-8e1a-83adb470564a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:49.013160 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef6502-16b3-4681-b126-cc6247556f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:49.014680 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef6502-16b3-4681-b126-cc6247556f55 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:49.015435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef6502-16b3-4681-b126-cc6247556f55 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:49.016112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef6502-16b3-4681-b126-cc6247556f55 None admin] PATH_INFO: `/v3/users/56bf68b0201e419781232cd507f5b7cb` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:49.018694 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef6502-16b3-4681-b126-cc6247556f55 None admin] RBAC: Authorizing `identity:get_user(user_id=56bf68b0201e419781232cd507f5b7cb)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:49.022132 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef6502-16b3-4681-b126-cc6247556f55 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:49.025570 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 113/232] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:48 2022] GET /identity/v3/users/56bf68b0201e419781232cd507f5b7cb => generated 283 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:49.035997 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e38242-90ee-485d-9fe5-ef128ed983dd None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:49.106100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:49.273053 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e38242-90ee-485d-9fe5-ef128ed983dd None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:49.359194 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31e473-2aa8-4650-89de-a37fcfa99d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:49.360524 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31e473-2aa8-4650-89de-a37fcfa99d80 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:49.361270 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31e473-2aa8-4650-89de-a37fcfa99d80 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:49.361996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31e473-2aa8-4650-89de-a37fcfa99d80 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:49.365113 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31e473-2aa8-4650-89de-a37fcfa99d80 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:49.368288 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31e473-2aa8-4650-89de-a37fcfa99d80 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:49.372463 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 120/233] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:49 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:49.381793 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef6502-16b3-4681-b126-cc6247556f55 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:49.597383 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef6502-16b3-4681-b126-cc6247556f55 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:49.689496 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c593d-de1c-4bff-a431-42d8a0852065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:49.690632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c593d-de1c-4bff-a431-42d8a0852065 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:49.691130 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c593d-de1c-4bff-a431-42d8a0852065 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:49.691559 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c593d-de1c-4bff-a431-42d8a0852065 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74/users/56bf68b0201e419781232cd507f5b7cb/roles/ca13aabb28b6442f8540fa4e8bb1b35a` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:49.700995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c593d-de1c-4bff-a431-42d8a0852065 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f97028a6a564e6b924279df149adb74, user_id=56bf68b0201e419781232cd507f5b7cb, role_id=ca13aabb28b6442f8540fa4e8bb1b35a)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:49.703954 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c593d-de1c-4bff-a431-42d8a0852065 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:49.746880 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 114/234] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:34:49 2022] PUT /identity/v3/projects/8f97028a6a564e6b924279df149adb74/users/56bf68b0201e419781232cd507f5b7cb/roles/ca13aabb28b6442f8540fa4e8bb1b35a => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:34:50.940141 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd727ff-e67c-4384-aa42-016de6c39016 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:50.940672 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd727ff-e67c-4384-aa42-016de6c39016 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:50.940672 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd727ff-e67c-4384-aa42-016de6c39016 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:50.942602 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 121/235] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:50.948297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cb8fd-d67c-42d7-91df-56202169ffc3 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:50.949428 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cb8fd-d67c-42d7-91df-56202169ffc3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:50.949428 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1cb8fd-d67c-42d7-91df-56202169ffc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:50.975054 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-7a1cb8fd-d67c-42d7-91df-56202169ffc3 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:50.976372 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7a1cb8fd-d67c-42d7-91df-56202169ffc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:51.001803 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:51.083254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 115/236] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:51.165873 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675f59c-9e7b-421a-b567-bed63fc9b0b4 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:51.166557 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675f59c-9e7b-421a-b567-bed63fc9b0b4 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:51.167157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675f59c-9e7b-421a-b567-bed63fc9b0b4 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:51.169187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 122/237] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:51.175572 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de610f4b-534b-4ef6-9034-0457880ec140 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:51.176214 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de610f4b-534b-4ef6-9034-0457880ec140 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:51.176681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de610f4b-534b-4ef6-9034-0457880ec140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:51.203289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-de610f4b-534b-4ef6-9034-0457880ec140 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:51.204825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-de610f4b-534b-4ef6-9034-0457880ec140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:51.314508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de219b13-196c-462a-9fa5-754523788662 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:51.314967 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de219b13-196c-462a-9fa5-754523788662 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:51.315422 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de219b13-196c-462a-9fa5-754523788662 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:51.318987 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 116/238] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:34:51.322640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 123/239] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:51.327458 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9baa3e-4adb-4cd0-b6b7-e8b51679a642 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:51.327735 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9baa3e-4adb-4cd0-b6b7-e8b51679a642 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:51.328190 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9baa3e-4adb-4cd0-b6b7-e8b51679a642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:51.353844 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-fa9baa3e-4adb-4cd0-b6b7-e8b51679a642 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:51.355121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-fa9baa3e-4adb-4cd0-b6b7-e8b51679a642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:51.447398 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de219b13-196c-462a-9fa5-754523788662 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:51.461181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 117/240] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:34:51.577319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:51.590793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de219b13-196c-462a-9fa5-754523788662 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:51.641223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42904128-caef-43fe-97e3-cb23fd87a87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:51.642354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42904128-caef-43fe-97e3-cb23fd87a87c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:51.642890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42904128-caef-43fe-97e3-cb23fd87a87c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:51.643378 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42904128-caef-43fe-97e3-cb23fd87a87c None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:51.646787 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42904128-caef-43fe-97e3-cb23fd87a87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:51.648539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42904128-caef-43fe-97e3-cb23fd87a87c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:51.650456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 124/241] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:51.658670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9baa3e-4adb-4cd0-b6b7-e8b51679a642 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:51.682601 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:51.802319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42904128-caef-43fe-97e3-cb23fd87a87c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:51.841205 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9baa3e-4adb-4cd0-b6b7-e8b51679a642 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:51.901836 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47cd2d-7b0c-45d0-bb80-7be2828d4c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:51.902794 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47cd2d-7b0c-45d0-bb80-7be2828d4c22 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:51.903080 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47cd2d-7b0c-45d0-bb80-7be2828d4c22 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:51.903378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47cd2d-7b0c-45d0-bb80-7be2828d4c22 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:51.904077 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47cd2d-7b0c-45d0-bb80-7be2828d4c22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:51.905591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47cd2d-7b0c-45d0-bb80-7be2828d4c22 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:51.957161 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 118/242] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:34:51 2022] POST /identity/v3/users => generated 293 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:34:51.958065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47cd2d-7b0c-45d0-bb80-7be2828d4c22 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:52.050281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42904128-caef-43fe-97e3-cb23fd87a87c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:52.084016 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:52.191922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47cd2d-7b0c-45d0-bb80-7be2828d4c22 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:52.199502 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9fda0-65ed-4e1b-b130-21250ada5be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:52.200721 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9fda0-65ed-4e1b-b130-21250ada5be1 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:52.201220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9fda0-65ed-4e1b-b130-21250ada5be1 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:52.201703 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9fda0-65ed-4e1b-b130-21250ada5be1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:52.228650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9fda0-65ed-4e1b-b130-21250ada5be1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-afa9fda0-65ed-4e1b-b130-21250ada5be1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:52.240385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:52.241306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:52.242115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:52.242115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:52.242115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:52.242115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:52.242115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:52.242115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:52.242115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:52.242115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:52.242115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:52.242115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:52.242115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:34:52.242115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:52.243267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 125/243] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:34:52.258958 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9fda0-65ed-4e1b-b130-21250ada5be1 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:52.280434 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614620e-2b7c-482b-b568-1be698879e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:52.281517 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614620e-2b7c-482b-b568-1be698879e6c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:52.281998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614620e-2b7c-482b-b568-1be698879e6c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:52.282515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614620e-2b7c-482b-b568-1be698879e6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:52.315856 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614620e-2b7c-482b-b568-1be698879e6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:52.326518 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-8614620e-2b7c-482b-b568-1be698879e6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:52.337293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:52.338258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:52.339042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:52.339042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:52.339042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:52.339042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:52.339042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:52.339042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:52.339042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:52.339042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:52.339042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:52.339042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:52.339042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:52.339042 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:52.339666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 119/244] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:52.342527 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967446cd-b3e9-4e87-bf57-f84877891997 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:52.343008 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967446cd-b3e9-4e87-bf57-f84877891997 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:52.343463 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967446cd-b3e9-4e87-bf57-f84877891997 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:52.347601 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 120/245] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:52.351044 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967446cd-b3e9-4e87-bf57-f84877891997 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:52.476115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9fda0-65ed-4e1b-b130-21250ada5be1 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:52.536491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86055733-9c34-47e2-ba33-91710c934308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:52.537556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86055733-9c34-47e2-ba33-91710c934308 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:52.538118 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86055733-9c34-47e2-ba33-91710c934308 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:52.538646 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86055733-9c34-47e2-ba33-91710c934308 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:52.539907 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86055733-9c34-47e2-ba33-91710c934308 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:52.540509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86055733-9c34-47e2-ba33-91710c934308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:52.543558 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86055733-9c34-47e2-ba33-91710c934308 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:52.556155 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967446cd-b3e9-4e87-bf57-f84877891997 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:52.563327 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 126/246] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:52.564746 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3ec42-753f-4171-9664-2321c2b175ba None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:52.565116 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3ec42-753f-4171-9664-2321c2b175ba None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:52.565396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3ec42-753f-4171-9664-2321c2b175ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:52.592945 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-d5e3ec42-753f-4171-9664-2321c2b175ba None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:52.594122 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-d5e3ec42-753f-4171-9664-2321c2b175ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:52.621410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:52.632051 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f562c-9939-456e-a02d-4d416d19c5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:52.632692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f562c-9939-456e-a02d-4d416d19c5df None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:52.632972 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f562c-9939-456e-a02d-4d416d19c5df None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:52.633246 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f562c-9939-456e-a02d-4d416d19c5df None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:52.633955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f562c-9939-456e-a02d-4d416d19c5df None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:52.634311 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f562c-9939-456e-a02d-4d416d19c5df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:52.635765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f562c-9939-456e-a02d-4d416d19c5df None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:52.648442 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 121/247] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:52.649354 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f562c-9939-456e-a02d-4d416d19c5df None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:52.741151 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 127/248] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:52.743238 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3ec42-753f-4171-9664-2321c2b175ba None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:52.794493 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:52.845529 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f562c-9939-456e-a02d-4d416d19c5df None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:52.920860 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ec481-eb2f-4f9c-877b-be6d4217c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:52.922168 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ec481-eb2f-4f9c-877b-be6d4217c34d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:52.922653 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ec481-eb2f-4f9c-877b-be6d4217c34d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:52.923123 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ec481-eb2f-4f9c-877b-be6d4217c34d None admin] PATH_INFO: `/v3/users/3d67e765fb7042a3898bdf46410073f4` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:52.959878 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3ec42-753f-4171-9664-2321c2b175ba None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:52.972074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ec481-eb2f-4f9c-877b-be6d4217c34d None admin] RBAC: Authorizing `identity:get_user(user_id=3d67e765fb7042a3898bdf46410073f4)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:52.974327 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ec481-eb2f-4f9c-877b-be6d4217c34d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:52.978064 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 122/249] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:52 2022] GET /identity/v3/users/3d67e765fb7042a3898bdf46410073f4 => generated 297 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:52.984951 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ec481-eb2f-4f9c-877b-be6d4217c34d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:53.032616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8f479-c7bf-4077-a963-0d787ea51502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:53.033684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8f479-c7bf-4077-a963-0d787ea51502 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:53.034194 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8f479-c7bf-4077-a963-0d787ea51502 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:53.034669 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8f479-c7bf-4077-a963-0d787ea51502 None admin] PATH_INFO: `/v3/users/25ee7eabd1d04f4381194934cb91b150` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:53.037147 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8f479-c7bf-4077-a963-0d787ea51502 None admin] RBAC: Authorizing `identity:get_user(user_id=25ee7eabd1d04f4381194934cb91b150)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:53.039888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8f479-c7bf-4077-a963-0d787ea51502 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:53.043536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 128/250] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:52 2022] GET /identity/v3/users/25ee7eabd1d04f4381194934cb91b150 => generated 297 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:53.052252 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8f479-c7bf-4077-a963-0d787ea51502 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:53.102210 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:53.208959 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ec481-eb2f-4f9c-877b-be6d4217c34d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:53.230460 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:53.264891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8f479-c7bf-4077-a963-0d787ea51502 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:53.332545 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ecd73-fb6b-4f65-b516-158f352e4b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:53.333625 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ecd73-fb6b-4f65-b516-158f352e4b29 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:53.334154 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ecd73-fb6b-4f65-b516-158f352e4b29 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:53.334649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ecd73-fb6b-4f65-b516-158f352e4b29 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:53.334915 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9a274-f448-4370-982c-1a13f093daa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:53.335924 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9a274-f448-4370-982c-1a13f093daa0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:53.336379 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9a274-f448-4370-982c-1a13f093daa0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:53.336821 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9a274-f448-4370-982c-1a13f093daa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:53.336951 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ecd73-fb6b-4f65-b516-158f352e4b29 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:53.340213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ecd73-fb6b-4f65-b516-158f352e4b29 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:53.343947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 129/251] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:53 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:53.353099 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ecd73-fb6b-4f65-b516-158f352e4b29 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:53.539134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9a274-f448-4370-982c-1a13f093daa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:53.541157 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9a274-f448-4370-982c-1a13f093daa0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:53.550901 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ecd73-fb6b-4f65-b516-158f352e4b29 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:53.630089 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cd9d2-ae8c-4368-ab9b-199c9ee2a312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:53.631208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cd9d2-ae8c-4368-ab9b-199c9ee2a312 None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:53.632132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cd9d2-ae8c-4368-ab9b-199c9ee2a312 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:53.632700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cd9d2-ae8c-4368-ab9b-199c9ee2a312 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74/users/25ee7eabd1d04f4381194934cb91b150/roles/d18e4897bf3648498a472a2ea8174f8b` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:53.636771 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cd9d2-ae8c-4368-ab9b-199c9ee2a312 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f97028a6a564e6b924279df149adb74, user_id=25ee7eabd1d04f4381194934cb91b150, role_id=d18e4897bf3648498a472a2ea8174f8b)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:53.640353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cd9d2-ae8c-4368-ab9b-199c9ee2a312 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:53.670576 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:53.686926 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 130/252] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:34:53 2022] PUT /identity/v3/projects/8f97028a6a564e6b924279df149adb74/users/25ee7eabd1d04f4381194934cb91b150/roles/d18e4897bf3648498a472a2ea8174f8b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:34:53.689256 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e383b-46de-4205-9cf0-22c27dbf80b9 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:53.689723 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e383b-46de-4205-9cf0-22c27dbf80b9 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:53.698448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e383b-46de-4205-9cf0-22c27dbf80b9 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:53.702233 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 131/253] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:53 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:53.710808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52b538-e829-425c-a8a0-7122e6da5738 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:53.711712 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52b538-e829-425c-a8a0-7122e6da5738 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:53.711712 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52b538-e829-425c-a8a0-7122e6da5738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:53.743607 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 123/254] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 759 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 12:34:53.752015 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9a274-f448-4370-982c-1a13f093daa0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:53.771870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-3d52b538-e829-425c-a8a0-7122e6da5738 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:53.773918 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-3d52b538-e829-425c-a8a0-7122e6da5738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:53.779520 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:53.894291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9a274-f448-4370-982c-1a13f093daa0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:53.947422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1adb0-1ae9-472e-8ebe-3fd287f7ba7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:53.948479 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1adb0-1ae9-472e-8ebe-3fd287f7ba7a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:53.949118 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1adb0-1ae9-472e-8ebe-3fd287f7ba7a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:53.949624 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1adb0-1ae9-472e-8ebe-3fd287f7ba7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:53.971604 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 132/255] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:53.974976 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52b538-e829-425c-a8a0-7122e6da5738 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:53.980021 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1adb0-1ae9-472e-8ebe-3fd287f7ba7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-7bb1adb0-1ae9-472e-8ebe-3fd287f7ba7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:53.986717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:53.987645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:53.988429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:53.988429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:53.988429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:53.988429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:53.988429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:53.988429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:53.988429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:53.988429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:53.988429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:53.988429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:53.988429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:34:53.988429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:54.001408 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 124/256] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:34:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:34:54.007436 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1adb0-1ae9-472e-8ebe-3fd287f7ba7a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:54.026192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:54.139056 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1adb0-1ae9-472e-8ebe-3fd287f7ba7a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:54.180872 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f8d7b-be8f-4e37-af53-3a94931393e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:54.181869 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f8d7b-be8f-4e37-af53-3a94931393e7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:54.182386 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f8d7b-be8f-4e37-af53-3a94931393e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:54.182864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f8d7b-be8f-4e37-af53-3a94931393e7 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:54.183808 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f8d7b-be8f-4e37-af53-3a94931393e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:54.184335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f8d7b-be8f-4e37-af53-3a94931393e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:54.185961 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f8d7b-be8f-4e37-af53-3a94931393e7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:54.201460 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 125/257] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:54.203359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f8d7b-be8f-4e37-af53-3a94931393e7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:54.288698 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:54.297880 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52b538-e829-425c-a8a0-7122e6da5738 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:54.328301 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f8d7b-be8f-4e37-af53-3a94931393e7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:54.368139 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cb72d-0823-4624-8ac0-2442549e5f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:54.368704 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabce353-19de-45a0-ae3c-9214d22b64b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:54.369534 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cb72d-0823-4624-8ac0-2442549e5f51 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:54.369705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabce353-19de-45a0-ae3c-9214d22b64b1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:54.370117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabce353-19de-45a0-ae3c-9214d22b64b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:54.370497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabce353-19de-45a0-ae3c-9214d22b64b1 None admin] PATH_INFO: `/v3/users/56bf68b0201e419781232cd507f5b7cb` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:54.370846 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cb72d-0823-4624-8ac0-2442549e5f51 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:54.371667 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cb72d-0823-4624-8ac0-2442549e5f51 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:54.372152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabce353-19de-45a0-ae3c-9214d22b64b1 None admin] RBAC: Authorizing `identity:get_user(user_id=56bf68b0201e419781232cd507f5b7cb)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:54.373847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabce353-19de-45a0-ae3c-9214d22b64b1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:54.374395 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cb72d-0823-4624-8ac0-2442549e5f51 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:54.375737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 126/258] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:54 2022] GET /identity/v3/users/56bf68b0201e419781232cd507f5b7cb => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:54.377548 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6cb72d-0823-4624-8ac0-2442549e5f51 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:54.381056 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 133/259] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:53 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:54.381994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabce353-19de-45a0-ae3c-9214d22b64b1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:54.388461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cb72d-0823-4624-8ac0-2442549e5f51 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:54.409680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:54.503961 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabce353-19de-45a0-ae3c-9214d22b64b1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:54.548430 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65f044-64f0-487d-b31c-8667defbc229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:54.556718 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65f044-64f0-487d-b31c-8667defbc229 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:54.557441 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65f044-64f0-487d-b31c-8667defbc229 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:54.559455 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65f044-64f0-487d-b31c-8667defbc229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:54.578664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:54.682144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cb72d-0823-4624-8ac0-2442549e5f51 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:54.719119 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65f044-64f0-487d-b31c-8667defbc229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:54.723685 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65f044-64f0-487d-b31c-8667defbc229 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:54.783608 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2db987-1b6e-439b-8edf-60a3d5478218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:54.784495 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2db987-1b6e-439b-8edf-60a3d5478218 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:54.785024 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2db987-1b6e-439b-8edf-60a3d5478218 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:54.785444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2db987-1b6e-439b-8edf-60a3d5478218 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:54.786365 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2db987-1b6e-439b-8edf-60a3d5478218 None admin] RBAC: Adding query filter params (role.id=1b0fe0772e714d1ba454d992ef8fa266, scope.project.id=8f97028a6a564e6b924279df149adb74, user.id=3d67e765fb7042a3898bdf46410073f4) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:54.786821 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2db987-1b6e-439b-8edf-60a3d5478218 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:54.788495 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2db987-1b6e-439b-8edf-60a3d5478218 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:54.820255 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 134/260] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:54 2022] GET /identity/v3/role_assignments?user.id=3d67e765fb7042a3898bdf46410073f4&scope.project.id=8f97028a6a564e6b924279df149adb74&role.id=1b0fe0772e714d1ba454d992ef8fa266 => generated 259 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:54.867750 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2db987-1b6e-439b-8edf-60a3d5478218 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:55.007619 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 127/261] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 12:34:55.021401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65f044-64f0-487d-b31c-8667defbc229 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:55.062707 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:55.102575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:55.116598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2db987-1b6e-439b-8edf-60a3d5478218 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:55.222927 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b3074-7ab3-4756-8de9-4912b021e9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:55.223726 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b3074-7ab3-4756-8de9-4912b021e9a1 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:55.224013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b3074-7ab3-4756-8de9-4912b021e9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:55.224287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b3074-7ab3-4756-8de9-4912b021e9a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:55.231947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65f044-64f0-487d-b31c-8667defbc229 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:55.255307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b3074-7ab3-4756-8de9-4912b021e9a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-9d0b3074-7ab3-4756-8de9-4912b021e9a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:55.261088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:55.262716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:55.263632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:55.263632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:55.263632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:55.263632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:55.263632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:55.263632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:55.263632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:55.263632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:55.263632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:55.263632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:55.263632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:34:55.263632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:55.272499 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 135/262] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:34:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:55.285955 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b3074-7ab3-4756-8de9-4912b021e9a1 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:55.322320 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd600e4-3168-4927-b82f-58faa6aa6e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:55.323331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd600e4-3168-4927-b82f-58faa6aa6e6e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:55.325202 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd600e4-3168-4927-b82f-58faa6aa6e6e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:55.325844 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd600e4-3168-4927-b82f-58faa6aa6e6e None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:55.328335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd600e4-3168-4927-b82f-58faa6aa6e6e None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:55.331721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd600e4-3168-4927-b82f-58faa6aa6e6e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:55.336131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 128/263] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:55 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:55.348258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd600e4-3168-4927-b82f-58faa6aa6e6e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:55.610019 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd600e4-3168-4927-b82f-58faa6aa6e6e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:55.613620 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b3074-7ab3-4756-8de9-4912b021e9a1 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:55.642083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:55.674703 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12bdc0-b396-45f4-97df-ddb5f667109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:55.675343 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12bdc0-b396-45f4-97df-ddb5f667109f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:55.675629 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12bdc0-b396-45f4-97df-ddb5f667109f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:55.675894 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12bdc0-b396-45f4-97df-ddb5f667109f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:55.676647 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12bdc0-b396-45f4-97df-ddb5f667109f None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=8f97028a6a564e6b924279df149adb74, user.id=56bf68b0201e419781232cd507f5b7cb) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:55.676956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12bdc0-b396-45f4-97df-ddb5f667109f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:55.678894 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12bdc0-b396-45f4-97df-ddb5f667109f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:55.688543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 129/264] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:55 2022] GET /identity/v3/role_assignments?user.id=56bf68b0201e419781232cd507f5b7cb&scope.project.id=8f97028a6a564e6b924279df149adb74&role.id=fc68d072459d4720b9666c93bd535d63 => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:55.730987 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:55.743270 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561d2d3-904b-4664-97e6-9c05a6320b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:55.744355 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561d2d3-904b-4664-97e6-9c05a6320b55 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:55.744852 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561d2d3-904b-4664-97e6-9c05a6320b55 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:55.745339 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561d2d3-904b-4664-97e6-9c05a6320b55 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:55.747282 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561d2d3-904b-4664-97e6-9c05a6320b55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:55.747838 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561d2d3-904b-4664-97e6-9c05a6320b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:55.750581 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561d2d3-904b-4664-97e6-9c05a6320b55 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:55.775706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 136/265] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:34:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 490 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:55.784613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12bdc0-b396-45f4-97df-ddb5f667109f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:55.932576 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12bdc0-b396-45f4-97df-ddb5f667109f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:55.984197 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a830d-5ef8-4b25-94a1-0be40b10f6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:55.985320 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a830d-5ef8-4b25-94a1-0be40b10f6e7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:55.985819 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a830d-5ef8-4b25-94a1-0be40b10f6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:55.986347 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a830d-5ef8-4b25-94a1-0be40b10f6e7 None admin] PATH_INFO: `/v3/users/56bf68b0201e419781232cd507f5b7cb` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:55.988767 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a830d-5ef8-4b25-94a1-0be40b10f6e7 None admin] RBAC: Authorizing `identity:get_user(user_id=56bf68b0201e419781232cd507f5b7cb)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:55.991761 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a830d-5ef8-4b25-94a1-0be40b10f6e7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:56.002712 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 130/266] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:55 2022] GET /identity/v3/users/56bf68b0201e419781232cd507f5b7cb => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:56.007860 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561d2d3-904b-4664-97e6-9c05a6320b55 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:56.274842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561d2d3-904b-4664-97e6-9c05a6320b55 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:56.361792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9185b5-8f11-4cb6-89e1-4b31d6a7c5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:56.362909 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9185b5-8f11-4cb6-89e1-4b31d6a7c5b8 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:56.363393 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9185b5-8f11-4cb6-89e1-4b31d6a7c5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:56.363877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9185b5-8f11-4cb6-89e1-4b31d6a7c5b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:56.510423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:56.532063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9185b5-8f11-4cb6-89e1-4b31d6a7c5b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:56.534556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9185b5-8f11-4cb6-89e1-4b31d6a7c5b8 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:56.770477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 137/267] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 764 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:56.778402 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a830d-5ef8-4b25-94a1-0be40b10f6e7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:56.853319 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:56.899214 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdabf38-1294-45d8-879a-2473f2a4337a None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:56.899613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdabf38-1294-45d8-879a-2473f2a4337a None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:56.900105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdabf38-1294-45d8-879a-2473f2a4337a None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:56.904592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 138/268] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:56.922618 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe602d-5368-432a-957a-8d33d4f90e71 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:56.923135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe602d-5368-432a-957a-8d33d4f90e71 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:56.923634 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe602d-5368-432a-957a-8d33d4f90e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:56.950497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a830d-5ef8-4b25-94a1-0be40b10f6e7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:56.984166 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-4fbe602d-5368-432a-957a-8d33d4f90e71 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:56.986106 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-4fbe602d-5368-432a-957a-8d33d4f90e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:56.998470 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2164e0-338e-443e-811f-d883f97d8045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:56.999490 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2164e0-338e-443e-811f-d883f97d8045 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:56.999991 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2164e0-338e-443e-811f-d883f97d8045 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:57.000474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2164e0-338e-443e-811f-d883f97d8045 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:57.001838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2164e0-338e-443e-811f-d883f97d8045 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:57.003548 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2164e0-338e-443e-811f-d883f97d8045 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:57.005449 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 131/269] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:56 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:57.023235 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2164e0-338e-443e-811f-d883f97d8045 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:57.132502 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 139/270] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:57.138064 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c272136-5fae-4352-ba52-14d33cf62ed4 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:57.138064 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c272136-5fae-4352-ba52-14d33cf62ed4 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:57.138064 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c272136-5fae-4352-ba52-14d33cf62ed4 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:57.140022 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 140/271] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:57.141709 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761dced-3c6b-4115-bfb4-f036bccee72c None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:57.142074 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761dced-3c6b-4115-bfb4-f036bccee72c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:57.142460 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761dced-3c6b-4115-bfb4-f036bccee72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:57.176460 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2164e0-338e-443e-811f-d883f97d8045 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:57.178146 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-a761dced-3c6b-4115-bfb4-f036bccee72c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:57.179423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-a761dced-3c6b-4115-bfb4-f036bccee72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:57.193919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:57.220856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:57.251318 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4218808-dbf2-47c7-9fd7-0fd33e94aadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:57.251950 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4218808-dbf2-47c7-9fd7-0fd33e94aadb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:57.252226 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4218808-dbf2-47c7-9fd7-0fd33e94aadb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:57.252497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4218808-dbf2-47c7-9fd7-0fd33e94aadb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:57.253243 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4218808-dbf2-47c7-9fd7-0fd33e94aadb None admin] RBAC: Adding query filter params (role.id=ca13aabb28b6442f8540fa4e8bb1b35a, scope.project.id=8f97028a6a564e6b924279df149adb74, user.id=56bf68b0201e419781232cd507f5b7cb) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:57.253547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4218808-dbf2-47c7-9fd7-0fd33e94aadb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:57.255074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4218808-dbf2-47c7-9fd7-0fd33e94aadb None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:57.273539 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 132/272] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:34:57 2022] GET /identity/v3/role_assignments?user.id=56bf68b0201e419781232cd507f5b7cb&scope.project.id=8f97028a6a564e6b924279df149adb74&role.id=ca13aabb28b6442f8540fa4e8bb1b35a => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:57.303132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 141/273] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:34:57.515786 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4218808-dbf2-47c7-9fd7-0fd33e94aadb None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:57.674882 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761dced-3c6b-4115-bfb4-f036bccee72c None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:57.743203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4218808-dbf2-47c7-9fd7-0fd33e94aadb None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:57.812409 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7679fcd-6b37-4a2e-a5ca-c6c3ba223000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:57.813444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7679fcd-6b37-4a2e-a5ca-c6c3ba223000 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:57.813867 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7679fcd-6b37-4a2e-a5ca-c6c3ba223000 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:57.814392 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7679fcd-6b37-4a2e-a5ca-c6c3ba223000 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:57.816057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761dced-3c6b-4115-bfb4-f036bccee72c None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:57.835939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7679fcd-6b37-4a2e-a5ca-c6c3ba223000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a7679fcd-6b37-4a2e-a5ca-c6c3ba223000 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:57.847032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:57.848017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:57.848799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 133/274] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:34:57.863511 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7679fcd-6b37-4a2e-a5ca-c6c3ba223000 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:57.866004 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2beee-3916-4b90-94da-b8f96cd3108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:57.866908 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2beee-3916-4b90-94da-b8f96cd3108a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:57.867352 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2beee-3916-4b90-94da-b8f96cd3108a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:57.867770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2beee-3916-4b90-94da-b8f96cd3108a None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:57.869332 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2beee-3916-4b90-94da-b8f96cd3108a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:57.872535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2beee-3916-4b90-94da-b8f96cd3108a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:57.874467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 142/275] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:34:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:34:57.880739 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2beee-3916-4b90-94da-b8f96cd3108a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:57.913576 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:57.918564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:58.008415 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7679fcd-6b37-4a2e-a5ca-c6c3ba223000 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:58.029799 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2beee-3916-4b90-94da-b8f96cd3108a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:58.053380 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf5815-1fee-4c50-a859-45b2f9202b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:58.054023 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf5815-1fee-4c50-a859-45b2f9202b13 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:58.054330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf5815-1fee-4c50-a859-45b2f9202b13 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:58.054603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf5815-1fee-4c50-a859-45b2f9202b13 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:58.055339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf5815-1fee-4c50-a859-45b2f9202b13 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:58.055648 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf5815-1fee-4c50-a859-45b2f9202b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:58.057365 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf5815-1fee-4c50-a859-45b2f9202b13 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:58.070219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 134/276] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:58.074990 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87e893-bac5-4cfa-b633-318fd55bffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:58.075623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87e893-bac5-4cfa-b633-318fd55bffff None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:58.075901 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87e893-bac5-4cfa-b633-318fd55bffff None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:58.076170 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87e893-bac5-4cfa-b633-318fd55bffff None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:58.077335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf5815-1fee-4c50-a859-45b2f9202b13 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:58.078458 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87e893-bac5-4cfa-b633-318fd55bffff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:58.079904 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87e893-bac5-4cfa-b633-318fd55bffff None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:58.118066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 143/277] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:34:57 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:34:58.118066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0a96e-52b2-4c66-ad99-ae33306b06d2 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:58.118066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0a96e-52b2-4c66-ad99-ae33306b06d2 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:58.118066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0a96e-52b2-4c66-ad99-ae33306b06d2 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:58.119889 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 144/278] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:58.129057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8470c6-7fe4-4d91-a6cf-28aeb67df9fd None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:58.129643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8470c6-7fe4-4d91-a6cf-28aeb67df9fd None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:58.130396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8470c6-7fe4-4d91-a6cf-28aeb67df9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:58.165665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-8f8470c6-7fe4-4d91-a6cf-28aeb67df9fd None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:58.167243 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-8f8470c6-7fe4-4d91-a6cf-28aeb67df9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:58.210657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:58.222274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf5815-1fee-4c50-a859-45b2f9202b13 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:58.260461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 145/279] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:34:58.269301 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ca9df-89f6-400b-a6c6-540df15a075d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:58.270275 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ca9df-89f6-400b-a6c6-540df15a075d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:58.270554 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ca9df-89f6-400b-a6c6-540df15a075d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:58.270830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ca9df-89f6-400b-a6c6-540df15a075d None admin] PATH_INFO: `/v3/users/88b5bbf6cace4740a17fdb9afe0291d7` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:58.287228 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ca9df-89f6-400b-a6c6-540df15a075d None admin] RBAC: Authorizing `identity:get_user(user_id=88b5bbf6cace4740a17fdb9afe0291d7)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:58.289031 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ca9df-89f6-400b-a6c6-540df15a075d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:58.291219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 135/280] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:58 2022] GET /identity/v3/users/88b5bbf6cace4740a17fdb9afe0291d7 => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:58.297825 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8470c6-7fe4-4d91-a6cf-28aeb67df9fd None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:58.311179 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:58.447733 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8470c6-7fe4-4d91-a6cf-28aeb67df9fd None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:58.491784 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcd0b9-26cb-4119-8b97-3f7c01e9f61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:58.492483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcd0b9-26cb-4119-8b97-3f7c01e9f61e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:58.492762 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcd0b9-26cb-4119-8b97-3f7c01e9f61e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:58.493041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcd0b9-26cb-4119-8b97-3f7c01e9f61e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:58.493781 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dcd0b9-26cb-4119-8b97-3f7c01e9f61e None admin] RBAC: Adding query filter params (role.id=d18e4897bf3648498a472a2ea8174f8b, user.id=88b5bbf6cace4740a17fdb9afe0291d7, scope.system=all) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:58.494207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dcd0b9-26cb-4119-8b97-3f7c01e9f61e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:58.495649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dcd0b9-26cb-4119-8b97-3f7c01e9f61e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:58.502126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 146/281] 10.222.0.33 () {56 vars in 1308 bytes} [Tue Oct 4 12:34:58 2022] GET /identity/v3/role_assignments?user.id=88b5bbf6cace4740a17fdb9afe0291d7&scope.system=all&role.id=d18e4897bf3648498a472a2ea8174f8b => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:58.545939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ca9df-89f6-400b-a6c6-540df15a075d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:58.683419 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ca9df-89f6-400b-a6c6-540df15a075d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:58.716482 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f7146-51b9-4d9a-8ab5-1a03fdda602f None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:58.716673 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f7146-51b9-4d9a-8ab5-1a03fdda602f None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:58.716954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f7146-51b9-4d9a-8ab5-1a03fdda602f None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:58.719058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 147/282] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:34:58.728590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9107477-785d-4198-8daf-397882d81ff6 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:58.729226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9107477-785d-4198-8daf-397882d81ff6 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:58.729723 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9107477-785d-4198-8daf-397882d81ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:58.733679 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08015c-e4c3-4c25-b792-5047d881fd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:58.734369 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08015c-e4c3-4c25-b792-5047d881fd42 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:58.734652 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08015c-e4c3-4c25-b792-5047d881fd42 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:58.734925 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08015c-e4c3-4c25-b792-5047d881fd42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:58.751753 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-a9107477-785d-4198-8daf-397882d81ff6 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:34:58.752693 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08015c-e4c3-4c25-b792-5047d881fd42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:58.753160 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-a9107477-785d-4198-8daf-397882d81ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0d08015c-e4c3-4c25-b792-5047d881fd42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:58.758083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:58.759039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:34:58.760273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 136/283] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:34:58.767334 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08015c-e4c3-4c25-b792-5047d881fd42 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:58.784309 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:58.914290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08015c-e4c3-4c25-b792-5047d881fd42 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:58.925568 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 148/284] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:34:58.972371 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cbd59-8b04-4f0d-9272-c0dc0024da11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:58.973527 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cbd59-8b04-4f0d-9272-c0dc0024da11 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:58.974059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cbd59-8b04-4f0d-9272-c0dc0024da11 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:58.974570 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cbd59-8b04-4f0d-9272-c0dc0024da11 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:58.975857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cbd59-8b04-4f0d-9272-c0dc0024da11 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:34:58.976402 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cbd59-8b04-4f0d-9272-c0dc0024da11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:58.979130 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cbd59-8b04-4f0d-9272-c0dc0024da11 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:59.000835 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 137/285] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:59.007367 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9107477-785d-4198-8daf-397882d81ff6 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:59.012180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:59.212960 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9107477-785d-4198-8daf-397882d81ff6 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:59.271282 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385c985-c344-4103-b484-724fc9bd2535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:59.271951 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c985-c344-4103-b484-724fc9bd2535 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:59.272227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c985-c344-4103-b484-724fc9bd2535 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:59.272497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385c985-c344-4103-b484-724fc9bd2535 None admin] PATH_INFO: `/v3/users/25ee7eabd1d04f4381194934cb91b150` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:59.279106 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385c985-c344-4103-b484-724fc9bd2535 None admin] RBAC: Authorizing `identity:get_user(user_id=25ee7eabd1d04f4381194934cb91b150)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:59.280698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385c985-c344-4103-b484-724fc9bd2535 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:59.283057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 149/286] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:34:59 2022] GET /identity/v3/users/25ee7eabd1d04f4381194934cb91b150 => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:34:59.288808 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cbd59-8b04-4f0d-9272-c0dc0024da11 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:59.355862 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385c985-c344-4103-b484-724fc9bd2535 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:59.478471 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:59.487958 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:59.493655 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385c985-c344-4103-b484-724fc9bd2535 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:59.541185 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd9014-a0e4-41cb-9221-fd4146c1bee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:59.542317 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd9014-a0e4-41cb-9221-fd4146c1bee6 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:59.543345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd9014-a0e4-41cb-9221-fd4146c1bee6 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:59.543820 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd9014-a0e4-41cb-9221-fd4146c1bee6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:59.567557 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd9014-a0e4-41cb-9221-fd4146c1bee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-87fd9014-a0e4-41cb-9221-fd4146c1bee6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:34:59.576823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:34:59.577756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:34:59.578578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:34:59.578578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:34:59.578578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:34:59.578578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:34:59.578578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:34:59.578578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:34:59.578578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:34:59.578578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:34:59.578578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:34:59.578578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:34:59.578578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:34:59.578578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:34:59.583918 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cbd59-8b04-4f0d-9272-c0dc0024da11 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:59.585498 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 150/287] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:34:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:34:59.591258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd9014-a0e4-41cb-9221-fd4146c1bee6 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:59.658764 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb32e5-d6fe-478a-a9c3-fdb38d313861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:34:59.659945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb32e5-d6fe-478a-a9c3-fdb38d313861 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:34:59.661097 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb32e5-d6fe-478a-a9c3-fdb38d313861 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:34:59.661749 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb32e5-d6fe-478a-a9c3-fdb38d313861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:34:59.764424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb32e5-d6fe-478a-a9c3-fdb38d313861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:34:59.766442 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb32e5-d6fe-478a-a9c3-fdb38d313861 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:34:59.847826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:34:59.853752 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd9014-a0e4-41cb-9221-fd4146c1bee6 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:34:59.885933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 138/288] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:34:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:34:59.894445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb32e5-d6fe-478a-a9c3-fdb38d313861 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:34:59.966424 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:00.002817 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0030c0-ca53-43aa-add8-9474eb7afc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:00.003549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0030c0-ca53-43aa-add8-9474eb7afc65 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:00.003729 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0030c0-ca53-43aa-add8-9474eb7afc65 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:00.004005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0030c0-ca53-43aa-add8-9474eb7afc65 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:00.004729 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0030c0-ca53-43aa-add8-9474eb7afc65 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:00.005016 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0030c0-ca53-43aa-add8-9474eb7afc65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:00.006549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0030c0-ca53-43aa-add8-9474eb7afc65 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:00.014116 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb32e5-d6fe-478a-a9c3-fdb38d313861 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:00.029456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 151/289] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:34:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:00.039123 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0030c0-ca53-43aa-add8-9474eb7afc65 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:00.061169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae30039-3f1d-4723-83d2-98dc3a09dbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:00.062120 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae30039-3f1d-4723-83d2-98dc3a09dbab None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:00.062555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae30039-3f1d-4723-83d2-98dc3a09dbab None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:00.062971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae30039-3f1d-4723-83d2-98dc3a09dbab None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:00.064320 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae30039-3f1d-4723-83d2-98dc3a09dbab None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:00.065957 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae30039-3f1d-4723-83d2-98dc3a09dbab None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:00.068149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 139/290] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:34:59 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:00.074546 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae30039-3f1d-4723-83d2-98dc3a09dbab None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:00.125388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:00.195147 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae30039-3f1d-4723-83d2-98dc3a09dbab None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:00.240971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696da0c-12cf-4dcf-a7c5-5893c0a041e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:00.241988 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696da0c-12cf-4dcf-a7c5-5893c0a041e8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:00.242500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696da0c-12cf-4dcf-a7c5-5893c0a041e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:00.242970 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696da0c-12cf-4dcf-a7c5-5893c0a041e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:00.243919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696da0c-12cf-4dcf-a7c5-5893c0a041e8 None admin] RBAC: Adding query filter params (role.id=d18e4897bf3648498a472a2ea8174f8b, scope.project.id=8f97028a6a564e6b924279df149adb74, user.id=25ee7eabd1d04f4381194934cb91b150) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:00.244472 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696da0c-12cf-4dcf-a7c5-5893c0a041e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:00.246246 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696da0c-12cf-4dcf-a7c5-5893c0a041e8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:00.253524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 140/291] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:35:00 2022] GET /identity/v3/role_assignments?user.id=25ee7eabd1d04f4381194934cb91b150&scope.project.id=8f97028a6a564e6b924279df149adb74&role.id=d18e4897bf3648498a472a2ea8174f8b => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:00.305505 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0030c0-ca53-43aa-add8-9474eb7afc65 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:00.360128 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded30b6e-0b8f-4b92-8be3-f46f1169830f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:00.360877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded30b6e-0b8f-4b92-8be3-f46f1169830f None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:00.361166 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded30b6e-0b8f-4b92-8be3-f46f1169830f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:00.361444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded30b6e-0b8f-4b92-8be3-f46f1169830f None admin] PATH_INFO: `/v3/users/3d67e765fb7042a3898bdf46410073f4` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:00.362759 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded30b6e-0b8f-4b92-8be3-f46f1169830f None admin] RBAC: Authorizing `identity:get_user(user_id=3d67e765fb7042a3898bdf46410073f4)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:00.364300 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded30b6e-0b8f-4b92-8be3-f46f1169830f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:00.366072 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 152/292] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:35:00 2022] GET /identity/v3/users/3d67e765fb7042a3898bdf46410073f4 => generated 297 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:00.375169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696da0c-12cf-4dcf-a7c5-5893c0a041e8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:00.547502 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696da0c-12cf-4dcf-a7c5-5893c0a041e8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:00.554979 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:00.612246 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13259911-d200-44f9-b955-27365fbda344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:00.612909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13259911-d200-44f9-b955-27365fbda344 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:00.613185 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13259911-d200-44f9-b955-27365fbda344 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:00.613458 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13259911-d200-44f9-b955-27365fbda344 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:00.614714 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13259911-d200-44f9-b955-27365fbda344 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:00.616217 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13259911-d200-44f9-b955-27365fbda344 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:00.618395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 141/293] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:00 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:00.629117 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded30b6e-0b8f-4b92-8be3-f46f1169830f None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:00.708236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:00.762488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded30b6e-0b8f-4b92-8be3-f46f1169830f None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:00.801963 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ca435-d31d-4893-97bc-56874001154d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:00.802611 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ca435-d31d-4893-97bc-56874001154d None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:00.802892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ca435-d31d-4893-97bc-56874001154d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:00.803165 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ca435-d31d-4893-97bc-56874001154d None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74/users/3d67e765fb7042a3898bdf46410073f4/roles/1b0fe0772e714d1ba454d992ef8fa266` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:00.805245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ca435-d31d-4893-97bc-56874001154d None admin] RBAC: Authorizing `identity:create_grant(project_id=8f97028a6a564e6b924279df149adb74, user_id=3d67e765fb7042a3898bdf46410073f4, role_id=1b0fe0772e714d1ba454d992ef8fa266)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:00.806856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ca435-d31d-4893-97bc-56874001154d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:00.826506 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 153/294] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:35:00 2022] PUT /identity/v3/projects/8f97028a6a564e6b924279df149adb74/users/3d67e765fb7042a3898bdf46410073f4/roles/1b0fe0772e714d1ba454d992ef8fa266 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:35:01.226998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a100dc1-62fa-4593-96a2-f5b5ed678031 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:01.228020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a100dc1-62fa-4593-96a2-f5b5ed678031 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:01.228635 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a100dc1-62fa-4593-96a2-f5b5ed678031 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:01.230713 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 142/295] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:01.236759 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df696eb-f8a2-44b2-bb4d-5e738fe1318e None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:01.237697 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df696eb-f8a2-44b2-bb4d-5e738fe1318e None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:01.238211 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df696eb-f8a2-44b2-bb4d-5e738fe1318e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:01.257860 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-2df696eb-f8a2-44b2-bb4d-5e738fe1318e None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:01.259500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-2df696eb-f8a2-44b2-bb4d-5e738fe1318e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:01.272437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab368e-0f81-4118-ae9a-79ac6f4199f3 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:01.272717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab368e-0f81-4118-ae9a-79ac6f4199f3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:01.273203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab368e-0f81-4118-ae9a-79ac6f4199f3 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:01.275294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 143/296] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:35:01.281529 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae43d9-ba37-460a-a2d0-6190eb8c967b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:01.282347 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae43d9-ba37-460a-a2d0-6190eb8c967b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:01.282819 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae43d9-ba37-460a-a2d0-6190eb8c967b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:01.311696 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-0eae43d9-ba37-460a-a2d0-6190eb8c967b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:01.313958 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0eae43d9-ba37-460a-a2d0-6190eb8c967b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:01.344761 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 154/297] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:01.402279 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 144/298] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:35:01.631749 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df696eb-f8a2-44b2-bb4d-5e738fe1318e None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:01.730848 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:01.766522 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae43d9-ba37-460a-a2d0-6190eb8c967b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:01.861481 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df696eb-f8a2-44b2-bb4d-5e738fe1318e None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:01.912234 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae43d9-ba37-460a-a2d0-6190eb8c967b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:01.917010 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:01.939741 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c5246-a167-479d-92b5-b2d8deb3a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:01.940847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c5246-a167-479d-92b5-b2d8deb3a0bc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:01.941363 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c5246-a167-479d-92b5-b2d8deb3a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:01.941844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c5246-a167-479d-92b5-b2d8deb3a0bc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:01.960141 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80fb35-6700-4960-9356-39eb3b694d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:01.961321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80fb35-6700-4960-9356-39eb3b694d58 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:01.961823 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80fb35-6700-4960-9356-39eb3b694d58 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:01.962680 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80fb35-6700-4960-9356-39eb3b694d58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:01.964072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80fb35-6700-4960-9356-39eb3b694d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:01.965816 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80fb35-6700-4960-9356-39eb3b694d58 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:01.967666 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c5246-a167-479d-92b5-b2d8deb3a0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:01.973218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 145/299] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:01.974526 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80fb35-6700-4960-9356-39eb3b694d58 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-589c5246-a167-479d-92b5-b2d8deb3a0bc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:01.978629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:01.979509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:01.980861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:01.980861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:01.980861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:01.980861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:01.980861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:01.980861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:01.980861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:01.980861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:01.980861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:01.980861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:01.980861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:35:01.980861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:01.981859 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 155/300] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:01.989465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c5246-a167-479d-92b5-b2d8deb3a0bc None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:02.131340 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80fb35-6700-4960-9356-39eb3b694d58 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:02.177152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a0e21-f7e9-47d1-a42f-4671edb50203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:02.178331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a0e21-f7e9-47d1-a42f-4671edb50203 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:02.178846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a0e21-f7e9-47d1-a42f-4671edb50203 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:02.179325 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a0e21-f7e9-47d1-a42f-4671edb50203 None admin] PATH_INFO: `/v3/groups` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:02.180182 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a0e21-f7e9-47d1-a42f-4671edb50203 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:02.182084 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a0e21-f7e9-47d1-a42f-4671edb50203 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:02.206829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 146/301] 10.222.0.33 () {60 vars in 1134 bytes} [Tue Oct 4 12:35:01 2022] POST /identity/v3/groups => generated 229 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:35:02.217708 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc24649-f3bd-455e-bdfb-35d4c02b853e None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:02.218248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc24649-f3bd-455e-bdfb-35d4c02b853e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:02.218740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc24649-f3bd-455e-bdfb-35d4c02b853e None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:02.227947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 147/302] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:02 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:02.231098 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfe4e6-7786-43f9-9e47-0f27ca53db7e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:02.231581 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfe4e6-7786-43f9-9e47-0f27ca53db7e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:02.232064 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfe4e6-7786-43f9-9e47-0f27ca53db7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:02.255054 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c5246-a167-479d-92b5-b2d8deb3a0bc None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:02.279475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:02.280944 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-bbdfe4e6-7786-43f9-9e47-0f27ca53db7e None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:02.283215 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bbdfe4e6-7786-43f9-9e47-0f27ca53db7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:02.329233 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83688ba0-ed3c-4d06-b28a-3975e2a685cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:02.330303 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83688ba0-ed3c-4d06-b28a-3975e2a685cc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:02.330961 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83688ba0-ed3c-4d06-b28a-3975e2a685cc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:02.331446 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83688ba0-ed3c-4d06-b28a-3975e2a685cc None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:02.332269 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:02.334802 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83688ba0-ed3c-4d06-b28a-3975e2a685cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:02.335364 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83688ba0-ed3c-4d06-b28a-3975e2a685cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:02.338412 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83688ba0-ed3c-4d06-b28a-3975e2a685cc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:02.366106 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 156/303] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:35:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:02.366615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83688ba0-ed3c-4d06-b28a-3975e2a685cc None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:02.457827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 148/304] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:35:02.556933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83688ba0-ed3c-4d06-b28a-3975e2a685cc None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:02.604370 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3313-fb52-445a-ac4c-562d0b4ab1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:02.605126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3313-fb52-445a-ac4c-562d0b4ab1d3 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:02.605405 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3313-fb52-445a-ac4c-562d0b4ab1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:02.605681 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3313-fb52-445a-ac4c-562d0b4ab1d3 None admin] PATH_INFO: `/v3/users/63e16848dd4543a39a644d2605808e38` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:02.623902 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae3313-fb52-445a-ac4c-562d0b4ab1d3 None admin] RBAC: Authorizing `identity:get_user(user_id=63e16848dd4543a39a644d2605808e38)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:02.626123 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae3313-fb52-445a-ac4c-562d0b4ab1d3 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:02.628240 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 157/305] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:35:02 2022] GET /identity/v3/users/63e16848dd4543a39a644d2605808e38 => generated 293 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:02.635137 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfe4e6-7786-43f9-9e47-0f27ca53db7e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:02.767846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfe4e6-7786-43f9-9e47-0f27ca53db7e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:02.807943 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f614eb-05ca-4f1f-a8ca-00963bfe2783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:02.808687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f614eb-05ca-4f1f-a8ca-00963bfe2783 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:02.808966 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f614eb-05ca-4f1f-a8ca-00963bfe2783 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:02.809242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f614eb-05ca-4f1f-a8ca-00963bfe2783 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:02.810055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f614eb-05ca-4f1f-a8ca-00963bfe2783 None admin] RBAC: Adding query filter params (role.id=1b0fe0772e714d1ba454d992ef8fa266, user.id=63e16848dd4543a39a644d2605808e38, scope.system=all) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:02.810377 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f614eb-05ca-4f1f-a8ca-00963bfe2783 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:02.811842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f614eb-05ca-4f1f-a8ca-00963bfe2783 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:02.818084 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 149/306] 10.222.0.33 () {56 vars in 1308 bytes} [Tue Oct 4 12:35:02 2022] GET /identity/v3/role_assignments?user.id=63e16848dd4543a39a644d2605808e38&scope.system=all&role.id=1b0fe0772e714d1ba454d992ef8fa266 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:02.972566 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3313-fb52-445a-ac4c-562d0b4ab1d3 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:03.024578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:03.162217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3313-fb52-445a-ac4c-562d0b4ab1d3 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:03.215040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3cbfe-7406-4c31-93f8-7fff3ed2641e None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:03.215678 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3cbfe-7406-4c31-93f8-7fff3ed2641e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:03.216160 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3cbfe-7406-4c31-93f8-7fff3ed2641e None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:03.222014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 150/307] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:03.224060 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e271091-6a27-446d-88b4-f3afc9cbaf78 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:03.224577 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e271091-6a27-446d-88b4-f3afc9cbaf78 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:03.225051 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e271091-6a27-446d-88b4-f3afc9cbaf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:03.225996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee12650-3033-466f-84ed-e87222918a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:03.227063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee12650-3033-466f-84ed-e87222918a36 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:03.227524 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee12650-3033-466f-84ed-e87222918a36 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:03.228006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee12650-3033-466f-84ed-e87222918a36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:03.247584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4e271091-6a27-446d-88b4-f3afc9cbaf78 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:03.248129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee12650-3033-466f-84ed-e87222918a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:03.249616 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4e271091-6a27-446d-88b4-f3afc9cbaf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-0ee12650-3033-466f-84ed-e87222918a36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:03.266877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:03.268145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:03.268937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:03.268937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:03.268937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:03.268937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:03.268937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:03.268937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:03.268937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:03.268937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:03.268937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:03.268937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:03.268937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:03.268937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:03.270104 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 158/308] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:03.279514 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee12650-3033-466f-84ed-e87222918a36 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:03.281241 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:03.358067 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 151/309] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:35:03.452679 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:03.472706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee12650-3033-466f-84ed-e87222918a36 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:03.536340 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e3164-a490-423c-adea-9680c9b4a3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:03.537294 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e3164-a490-423c-adea-9680c9b4a3ff None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:03.537789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e3164-a490-423c-adea-9680c9b4a3ff None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:03.538318 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e3164-a490-423c-adea-9680c9b4a3ff None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:03.539551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e3164-a490-423c-adea-9680c9b4a3ff None admin] RBAC: Adding query filter params (name=member) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:03.540113 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e3164-a490-423c-adea-9680c9b4a3ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:03.542918 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e3164-a490-423c-adea-9680c9b4a3ff None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:03.568085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e271091-6a27-446d-88b4-f3afc9cbaf78 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:03.574503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 159/310] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:35:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:03.666605 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e3164-a490-423c-adea-9680c9b4a3ff None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:03.697238 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e271091-6a27-446d-88b4-f3afc9cbaf78 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:03.740264 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112abb3-c83b-415a-892a-ffd180ade4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:03.741321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112abb3-c83b-415a-892a-ffd180ade4be None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:03.741843 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112abb3-c83b-415a-892a-ffd180ade4be None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:03.742365 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112abb3-c83b-415a-892a-ffd180ade4be None admin] PATH_INFO: `/v3/users/88b5bbf6cace4740a17fdb9afe0291d7` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:03.743766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112abb3-c83b-415a-892a-ffd180ade4be None admin] RBAC: Authorizing `identity:get_user(user_id=88b5bbf6cace4740a17fdb9afe0291d7)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:03.745477 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112abb3-c83b-415a-892a-ffd180ade4be None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:03.747361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 152/311] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:35:03 2022] GET /identity/v3/users/88b5bbf6cace4740a17fdb9afe0291d7 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:03.756650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112abb3-c83b-415a-892a-ffd180ade4be None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:03.784640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e3164-a490-423c-adea-9680c9b4a3ff None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:03.803605 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:03.827458 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d6ecf-3145-41f5-9940-a08b3ddcece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:03.828523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6ecf-3145-41f5-9940-a08b3ddcece0 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:03.828971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6ecf-3145-41f5-9940-a08b3ddcece0 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:03.829407 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6ecf-3145-41f5-9940-a08b3ddcece0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:03.842629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d6ecf-3145-41f5-9940-a08b3ddcece0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-713d6ecf-3145-41f5-9940-a08b3ddcece0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:03.848621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:03.849542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:03.850345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:03.850345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:03.850345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:03.850345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:03.850345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:03.850345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:03.850345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:03.850345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:03.850345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:03.850345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:03.850345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:35:03.850345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:03.854073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 160/312] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:03.863218 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d6ecf-3145-41f5-9940-a08b3ddcece0 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:03.882448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:03.940568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112abb3-c83b-415a-892a-ffd180ade4be None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:03.984915 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d8cc8-3a53-469a-8dd2-3c680a3d8cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:03.985903 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d8cc8-3a53-469a-8dd2-3c680a3d8cf4 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:03.986438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d8cc8-3a53-469a-8dd2-3c680a3d8cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:03.986879 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d8cc8-3a53-469a-8dd2-3c680a3d8cf4 None admin] PATH_INFO: `/v3/system/users/88b5bbf6cace4740a17fdb9afe0291d7/roles/d18e4897bf3648498a472a2ea8174f8b` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:03.988916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d8cc8-3a53-469a-8dd2-3c680a3d8cf4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=88b5bbf6cace4740a17fdb9afe0291d7, role_id=d18e4897bf3648498a472a2ea8174f8b)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:03.990661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d8cc8-3a53-469a-8dd2-3c680a3d8cf4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:04.010398 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 153/313] 10.222.0.33 () {58 vars in 1334 bytes} [Tue Oct 4 12:35:03 2022] PUT /identity/v3/system/users/88b5bbf6cace4740a17fdb9afe0291d7/roles/d18e4897bf3648498a472a2ea8174f8b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:35:04.034571 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d6ecf-3145-41f5-9940-a08b3ddcece0 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:04.088347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114c1e2-edf4-4c25-9e35-61172b3fcabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:04.088916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114c1e2-edf4-4c25-9e35-61172b3fcabc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:04.089316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114c1e2-edf4-4c25-9e35-61172b3fcabc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:04.089662 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114c1e2-edf4-4c25-9e35-61172b3fcabc None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:04.091060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114c1e2-edf4-4c25-9e35-61172b3fcabc None admin] RBAC: Adding query filter params (name=reader) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:04.091509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114c1e2-edf4-4c25-9e35-61172b3fcabc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:04.093424 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114c1e2-edf4-4c25-9e35-61172b3fcabc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:04.111233 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 161/314] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:35:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:04.114943 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d8cc8-3a53-469a-8dd2-3c680a3d8cf4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:04.219644 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:04.262794 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d8cc8-3a53-469a-8dd2-3c680a3d8cf4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:04.318850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf63647-6ca7-4175-a0c8-137d6412356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:04.320487 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf63647-6ca7-4175-a0c8-137d6412356f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:04.321237 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf63647-6ca7-4175-a0c8-137d6412356f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:04.322076 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf63647-6ca7-4175-a0c8-137d6412356f None admin] PATH_INFO: `/v3/users/3d67e765fb7042a3898bdf46410073f4` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:04.324640 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf63647-6ca7-4175-a0c8-137d6412356f None admin] RBAC: Authorizing `identity:get_user(user_id=3d67e765fb7042a3898bdf46410073f4)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:04.327842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf63647-6ca7-4175-a0c8-137d6412356f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:04.330916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 154/315] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:35:04 2022] GET /identity/v3/users/3d67e765fb7042a3898bdf46410073f4 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:04.338129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114c1e2-edf4-4c25-9e35-61172b3fcabc None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:04.447085 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:04.486143 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114c1e2-edf4-4c25-9e35-61172b3fcabc None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:04.547991 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72783c4-1ec4-4896-babe-3cd3ca31302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:04.549478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72783c4-1ec4-4896-babe-3cd3ca31302c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:04.550317 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72783c4-1ec4-4896-babe-3cd3ca31302c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:04.551149 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72783c4-1ec4-4896-babe-3cd3ca31302c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:04.636127 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72783c4-1ec4-4896-babe-3cd3ca31302c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:04.637637 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72783c4-1ec4-4896-babe-3cd3ca31302c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:04.725066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:04.756100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 162/316] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:35:04.762401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf63647-6ca7-4175-a0c8-137d6412356f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:04.846678 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916e80a-8594-40e1-9eed-df61f03b62db None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:04.847005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916e80a-8594-40e1-9eed-df61f03b62db None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:04.847228 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916e80a-8594-40e1-9eed-df61f03b62db None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:04.848861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 163/317] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:04.854543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00312b62-5f39-4797-ab70-c00c7c5f19eb None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:04.854885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00312b62-5f39-4797-ab70-c00c7c5f19eb None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:04.855249 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00312b62-5f39-4797-ab70-c00c7c5f19eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:04.877839 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-00312b62-5f39-4797-ab70-c00c7c5f19eb None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:04.878299 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf63647-6ca7-4175-a0c8-137d6412356f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:04.879082 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-00312b62-5f39-4797-ab70-c00c7c5f19eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:04.920415 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec2e5c-8146-4e13-b4c7-a08deaa7df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:04.921458 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec2e5c-8146-4e13-b4c7-a08deaa7df46 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:04.921738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec2e5c-8146-4e13-b4c7-a08deaa7df46 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:04.922317 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec2e5c-8146-4e13-b4c7-a08deaa7df46 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:04.923523 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec2e5c-8146-4e13-b4c7-a08deaa7df46 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:04.924983 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec2e5c-8146-4e13-b4c7-a08deaa7df46 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:04.927045 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 155/318] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:04 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:04.933111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec2e5c-8146-4e13-b4c7-a08deaa7df46 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:04.960762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:04.998233 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 164/319] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:05.086601 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec2e5c-8146-4e13-b4c7-a08deaa7df46 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:05.135306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1db9d7-ca8e-4081-8692-98a14e0f4f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:05.135930 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1db9d7-ca8e-4081-8692-98a14e0f4f6b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:05.136841 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1db9d7-ca8e-4081-8692-98a14e0f4f6b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:05.137120 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1db9d7-ca8e-4081-8692-98a14e0f4f6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:05.137920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1db9d7-ca8e-4081-8692-98a14e0f4f6b None admin] RBAC: Adding query filter params (role.id=1b0fe0772e714d1ba454d992ef8fa266, scope.project.id=8f97028a6a564e6b924279df149adb74, user.id=3d67e765fb7042a3898bdf46410073f4) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:05.138261 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1db9d7-ca8e-4081-8692-98a14e0f4f6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:05.139819 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1db9d7-ca8e-4081-8692-98a14e0f4f6b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:05.154294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 156/320] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:35:04 2022] GET /identity/v3/role_assignments?user.id=3d67e765fb7042a3898bdf46410073f4&scope.project.id=8f97028a6a564e6b924279df149adb74&role.id=1b0fe0772e714d1ba454d992ef8fa266 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:05.297319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00312b62-5f39-4797-ab70-c00c7c5f19eb None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:05.451530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00312b62-5f39-4797-ab70-c00c7c5f19eb None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:05.454557 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:05.500344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a554dab-1208-449b-bb0a-6f09a4dc8160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:05.501152 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a554dab-1208-449b-bb0a-6f09a4dc8160 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:05.501465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a554dab-1208-449b-bb0a-6f09a4dc8160 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:05.501713 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a554dab-1208-449b-bb0a-6f09a4dc8160 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:05.503433 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a554dab-1208-449b-bb0a-6f09a4dc8160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:05.506365 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a554dab-1208-449b-bb0a-6f09a4dc8160 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:05.509696 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 165/321] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:05.515270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1db9d7-ca8e-4081-8692-98a14e0f4f6b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:05.625666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:05.644769 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1db9d7-ca8e-4081-8692-98a14e0f4f6b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:05.685912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53baaa-4130-4049-8670-63f428c2f8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:05.686687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53baaa-4130-4049-8670-63f428c2f8aa None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:05.686962 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53baaa-4130-4049-8670-63f428c2f8aa None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:05.687242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53baaa-4130-4049-8670-63f428c2f8aa None admin] PATH_INFO: `/v3/groups` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:05.688115 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53baaa-4130-4049-8670-63f428c2f8aa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:05.689927 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53baaa-4130-4049-8670-63f428c2f8aa None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:05.692395 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6f0e4-10c8-4af8-95f3-7f79e2305a34 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:05.692676 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6f0e4-10c8-4af8-95f3-7f79e2305a34 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:05.692945 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6f0e4-10c8-4af8-95f3-7f79e2305a34 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:05.694500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 166/322] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:35:05.699877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aac47-884f-40a1-9d67-7f007496337b None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:05.700297 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aac47-884f-40a1-9d67-7f007496337b None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:05.700567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aac47-884f-40a1-9d67-7f007496337b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:05.711026 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 157/323] 10.222.0.33 () {60 vars in 1134 bytes} [Tue Oct 4 12:35:05 2022] POST /identity/v3/groups => generated 226 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:35:05.720088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-5f9aac47-884f-40a1-9d67-7f007496337b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:05.721305 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-5f9aac47-884f-40a1-9d67-7f007496337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:05.813057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 167/324] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:06.084282 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53baaa-4130-4049-8670-63f428c2f8aa None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:06.294778 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53baaa-4130-4049-8670-63f428c2f8aa None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:06.354787 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad41f7a-6f2b-43d4-a4bd-1bd3a5b435a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:06.355383 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad41f7a-6f2b-43d4-a4bd-1bd3a5b435a3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:06.355710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad41f7a-6f2b-43d4-a4bd-1bd3a5b435a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:06.355971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad41f7a-6f2b-43d4-a4bd-1bd3a5b435a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:06.369378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad41f7a-6f2b-43d4-a4bd-1bd3a5b435a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-7ad41f7a-6f2b-43d4-a4bd-1bd3a5b435a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:06.375034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:06.376919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:06.378675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 158/325] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:06.385177 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aac47-884f-40a1-9d67-7f007496337b None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:06.452764 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfb367-3da0-4985-a2e4-6e0e28ab0e57 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:06.453156 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfb367-3da0-4985-a2e4-6e0e28ab0e57 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:06.453313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfb367-3da0-4985-a2e4-6e0e28ab0e57 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:06.455002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 159/326] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:35:06.463986 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde1b79-7fd5-45d7-8486-b5475638f981 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:06.465435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde1b79-7fd5-45d7-8486-b5475638f981 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:06.466338 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde1b79-7fd5-45d7-8486-b5475638f981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:06.490699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:06.501789 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1fde1b79-7fd5-45d7-8486-b5475638f981 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:06.503977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1fde1b79-7fd5-45d7-8486-b5475638f981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:06.588476 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aac47-884f-40a1-9d67-7f007496337b None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:06.591612 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:06.654177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 160/327] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:35:06.657911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc73b5-caa4-4cbe-8282-93abfbaeaeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:06.658992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc73b5-caa4-4cbe-8282-93abfbaeaeed None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:06.659466 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc73b5-caa4-4cbe-8282-93abfbaeaeed None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:06.659936 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc73b5-caa4-4cbe-8282-93abfbaeaeed None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:06.661127 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc73b5-caa4-4cbe-8282-93abfbaeaeed None admin] RBAC: Adding query filter params (name=reader) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:06.661652 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc73b5-caa4-4cbe-8282-93abfbaeaeed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:06.664283 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc73b5-caa4-4cbe-8282-93abfbaeaeed None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:06.683735 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 168/328] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:35:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:06.689946 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde1b79-7fd5-45d7-8486-b5475638f981 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:06.882288 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde1b79-7fd5-45d7-8486-b5475638f981 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:06.944657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d96ec-38fc-4689-8c45-b32cf19c2e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:06.945261 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d96ec-38fc-4689-8c45-b32cf19c2e9a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:06.945543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d96ec-38fc-4689-8c45-b32cf19c2e9a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:06.945947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d96ec-38fc-4689-8c45-b32cf19c2e9a None admin] PATH_INFO: `/v3/users/63e16848dd4543a39a644d2605808e38` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:06.951061 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d96ec-38fc-4689-8c45-b32cf19c2e9a None admin] RBAC: Authorizing `identity:get_user(user_id=63e16848dd4543a39a644d2605808e38)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:06.952611 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d96ec-38fc-4689-8c45-b32cf19c2e9a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:06.954776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 161/329] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:35:06 2022] GET /identity/v3/users/63e16848dd4543a39a644d2605808e38 => generated 293 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:06.960992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc73b5-caa4-4cbe-8282-93abfbaeaeed None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:07.137284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc73b5-caa4-4cbe-8282-93abfbaeaeed None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:07.181105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a21e2-cdd8-40f9-97e7-01054c9ad703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:07.181769 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a21e2-cdd8-40f9-97e7-01054c9ad703 None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:07.182070 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a21e2-cdd8-40f9-97e7-01054c9ad703 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:07.182337 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a21e2-cdd8-40f9-97e7-01054c9ad703 None admin] PATH_INFO: `/v3/system/users/63e16848dd4543a39a644d2605808e38/roles/1b0fe0772e714d1ba454d992ef8fa266` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:07.186829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a21e2-cdd8-40f9-97e7-01054c9ad703 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=63e16848dd4543a39a644d2605808e38, role_id=1b0fe0772e714d1ba454d992ef8fa266)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:07.188354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a21e2-cdd8-40f9-97e7-01054c9ad703 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:07.201627 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 169/330] 10.222.0.33 () {58 vars in 1334 bytes} [Tue Oct 4 12:35:06 2022] PUT /identity/v3/system/users/63e16848dd4543a39a644d2605808e38/roles/1b0fe0772e714d1ba454d992ef8fa266 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:35:07.227196 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d96ec-38fc-4689-8c45-b32cf19c2e9a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:07.231825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:07.432713 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d96ec-38fc-4689-8c45-b32cf19c2e9a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:07.498349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a72d3-0242-4be6-8720-de12a93876ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:07.499305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a72d3-0242-4be6-8720-de12a93876ee None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:07.499722 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a72d3-0242-4be6-8720-de12a93876ee None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:07.500187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a72d3-0242-4be6-8720-de12a93876ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:07.520630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a72d3-0242-4be6-8720-de12a93876ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-dd1a72d3-0242-4be6-8720-de12a93876ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:07.530146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:07.531249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:07.532467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:07.532467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:07.532467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:07.532467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:07.532467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:07.532467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:07.532467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:07.532467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:07.532467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:07.532467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:07.532467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:07.532467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:07.533787 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 162/331] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:07.540934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a21e2-cdd8-40f9-97e7-01054c9ad703 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:07.587922 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:07.684832 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a21e2-cdd8-40f9-97e7-01054c9ad703 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:07.737789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe7b87-2898-4914-946c-fc778fd37d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:07.738771 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe7b87-2898-4914-946c-fc778fd37d22 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:07.739216 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe7b87-2898-4914-946c-fc778fd37d22 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:07.739735 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe7b87-2898-4914-946c-fc778fd37d22 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:07.740562 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe7b87-2898-4914-946c-fc778fd37d22 None admin] RBAC: Adding query filter params (name=member) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:07.741016 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe7b87-2898-4914-946c-fc778fd37d22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:07.742683 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe7b87-2898-4914-946c-fc778fd37d22 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:07.758740 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 170/332] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:35:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:07.765885 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a72d3-0242-4be6-8720-de12a93876ee None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:07.908528 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:07.997498 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a72d3-0242-4be6-8720-de12a93876ee None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:08.072176 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80439b-7f62-40bc-834d-0109ff0bc3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:08.073081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80439b-7f62-40bc-834d-0109ff0bc3de None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:08.074079 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80439b-7f62-40bc-834d-0109ff0bc3de None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:08.074545 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80439b-7f62-40bc-834d-0109ff0bc3de None admin] PATH_INFO: `/v3/users/88b5bbf6cace4740a17fdb9afe0291d7` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:08.076746 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80439b-7f62-40bc-834d-0109ff0bc3de None admin] RBAC: Authorizing `identity:get_user(user_id=88b5bbf6cace4740a17fdb9afe0291d7)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:08.083622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80439b-7f62-40bc-834d-0109ff0bc3de None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:08.087310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 163/333] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:35:07 2022] GET /identity/v3/users/88b5bbf6cace4740a17fdb9afe0291d7 => generated 293 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:08.094325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe7b87-2898-4914-946c-fc778fd37d22 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:08.234290 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:08.248595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe7b87-2898-4914-946c-fc778fd37d22 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:08.294808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e90b0-9cca-470a-ad92-47f3821361e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:08.295445 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e90b0-9cca-470a-ad92-47f3821361e7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:08.295724 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e90b0-9cca-470a-ad92-47f3821361e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:08.295996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e90b0-9cca-470a-ad92-47f3821361e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:08.296808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e90b0-9cca-470a-ad92-47f3821361e7 None admin] RBAC: Adding query filter params (role.id=d18e4897bf3648498a472a2ea8174f8b, user.id=88b5bbf6cace4740a17fdb9afe0291d7, scope.system=all) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:08.297123 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e90b0-9cca-470a-ad92-47f3821361e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:08.298648 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e90b0-9cca-470a-ad92-47f3821361e7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:08.317997 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 171/334] 10.222.0.33 () {56 vars in 1308 bytes} [Tue Oct 4 12:35:08 2022] GET /identity/v3/role_assignments?user.id=88b5bbf6cace4740a17fdb9afe0291d7&scope.system=all&role.id=d18e4897bf3648498a472a2ea8174f8b => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:09.527279 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54d914-0246-4629-bd35-195c79a3108b None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:09.527767 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54d914-0246-4629-bd35-195c79a3108b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:09.528360 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54d914-0246-4629-bd35-195c79a3108b None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:09.530991 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 164/335] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:09.542033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829812e6-b2c0-4a7e-9c4d-2a94c32edf57 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:09.543109 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829812e6-b2c0-4a7e-9c4d-2a94c32edf57 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:09.543560 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829812e6-b2c0-4a7e-9c4d-2a94c32edf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:09.579086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-829812e6-b2c0-4a7e-9c4d-2a94c32edf57 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:09.581101 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-829812e6-b2c0-4a7e-9c4d-2a94c32edf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:09.731342 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 172/336] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:10.300868 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54d914-0246-4629-bd35-195c79a3108b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:10.439459 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54d914-0246-4629-bd35-195c79a3108b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:10.474594 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:10.481666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04df29e-17b9-43ac-95d3-fc07aeea010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:10.482342 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04df29e-17b9-43ac-95d3-fc07aeea010c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:10.482649 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04df29e-17b9-43ac-95d3-fc07aeea010c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:10.482903 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04df29e-17b9-43ac-95d3-fc07aeea010c None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:10.500419 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04df29e-17b9-43ac-95d3-fc07aeea010c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-b04df29e-17b9-43ac-95d3-fc07aeea010c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:10.505348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:10.506268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:10.507094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 165/337] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:10.515190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829812e6-b2c0-4a7e-9c4d-2a94c32edf57 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:10.708968 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:10.727731 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bd55f-734a-4f03-a357-2b265119acf0 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:10.728728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bd55f-734a-4f03-a357-2b265119acf0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:10.729209 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bd55f-734a-4f03-a357-2b265119acf0 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:10.734736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 166/338] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:35:10.745188 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27f3e0-f5f0-4957-8000-5852fea06af7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:10.746421 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27f3e0-f5f0-4957-8000-5852fea06af7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:10.746947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27f3e0-f5f0-4957-8000-5852fea06af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:10.767659 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829812e6-b2c0-4a7e-9c4d-2a94c32edf57 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:10.779219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-0a27f3e0-f5f0-4957-8000-5852fea06af7 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:10.780810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0a27f3e0-f5f0-4957-8000-5852fea06af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:10.838212 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ade2f-056f-4f50-b892-0fa6f107a5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:10.839756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ade2f-056f-4f50-b892-0fa6f107a5dc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:10.840183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ade2f-056f-4f50-b892-0fa6f107a5dc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:10.840592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ade2f-056f-4f50-b892-0fa6f107a5dc None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:10.841551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ade2f-056f-4f50-b892-0fa6f107a5dc None admin] RBAC: Adding query filter params (name=member) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:10.842011 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ade2f-056f-4f50-b892-0fa6f107a5dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:10.843720 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ade2f-056f-4f50-b892-0fa6f107a5dc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:10.860724 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 173/339] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:35:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:10.866478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ade2f-056f-4f50-b892-0fa6f107a5dc None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:10.922977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 167/340] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:35:11.043476 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ade2f-056f-4f50-b892-0fa6f107a5dc None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:11.091135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52ab44-b49a-40ea-b9d3-635835e49456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:11.092031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52ab44-b49a-40ea-b9d3-635835e49456 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:11.092535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52ab44-b49a-40ea-b9d3-635835e49456 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:11.092947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52ab44-b49a-40ea-b9d3-635835e49456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:11.110114 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:11.190946 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52ab44-b49a-40ea-b9d3-635835e49456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:11.192692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52ab44-b49a-40ea-b9d3-635835e49456 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:11.338736 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 174/341] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 12:35:11.345326 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27f3e0-f5f0-4957-8000-5852fea06af7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:11.360425 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ca2fe-42dc-4f1d-9f7a-55fac1833325 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:11.360926 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ca2fe-42dc-4f1d-9f7a-55fac1833325 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:11.361404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ca2fe-42dc-4f1d-9f7a-55fac1833325 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:11.364538 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 175/342] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:11.375108 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88a1bb-cf37-46f2-a7ba-46383b9bef7a None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:11.376058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88a1bb-cf37-46f2-a7ba-46383b9bef7a None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:11.376823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88a1bb-cf37-46f2-a7ba-46383b9bef7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:11.414417 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-3e88a1bb-cf37-46f2-a7ba-46383b9bef7a None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:11.417428 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-3e88a1bb-cf37-46f2-a7ba-46383b9bef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:11.501784 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:11.553069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27f3e0-f5f0-4957-8000-5852fea06af7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:11.574349 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 176/343] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:11.576343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88a1bb-cf37-46f2-a7ba-46383b9bef7a None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:11.624712 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:11.674880 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4cafb-42a6-40e1-8423-be5e3b331c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:11.675543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4cafb-42a6-40e1-8423-be5e3b331c39 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:11.675813 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4cafb-42a6-40e1-8423-be5e3b331c39 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:11.676254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4cafb-42a6-40e1-8423-be5e3b331c39 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:11.678335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4cafb-42a6-40e1-8423-be5e3b331c39 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:11.680220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4cafb-42a6-40e1-8423-be5e3b331c39 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:11.683907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 168/344] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:11 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:11.688489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4cafb-42a6-40e1-8423-be5e3b331c39 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:11.864741 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88a1bb-cf37-46f2-a7ba-46383b9bef7a None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:11.939321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4cafb-42a6-40e1-8423-be5e3b331c39 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:11.944624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e9375-126b-4df2-b3dd-2ecae9e548c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:11.946642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e9375-126b-4df2-b3dd-2ecae9e548c9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:11.947511 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e9375-126b-4df2-b3dd-2ecae9e548c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:11.948277 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e9375-126b-4df2-b3dd-2ecae9e548c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:11.972964 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e9375-126b-4df2-b3dd-2ecae9e548c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-654e9375-126b-4df2-b3dd-2ecae9e548c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:11.984253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:11.986000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:11.987556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:11.987556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:11.987556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:11.987556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:11.987556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:11.987556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:11.987556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:11.987556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:11.987556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:11.987556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:11.987556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:11.987556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:11.992304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 177/345] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:11.999385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e9375-126b-4df2-b3dd-2ecae9e548c9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:12.041517 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d684a-fbcb-4a13-ada9-545a1545e3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:12.044817 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d684a-fbcb-4a13-ada9-545a1545e3bc None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:12.046629 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d684a-fbcb-4a13-ada9-545a1545e3bc None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:12.048279 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d684a-fbcb-4a13-ada9-545a1545e3bc None admin] PATH_INFO: `/v3/groups/58975584643c4b44a79722ad8c2ba30c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:12.074765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d684a-fbcb-4a13-ada9-545a1545e3bc None admin] RBAC: Authorizing `identity:get_group(group_id=58975584643c4b44a79722ad8c2ba30c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:12.082950 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d684a-fbcb-4a13-ada9-545a1545e3bc None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:12.090949 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 169/346] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:11 2022] GET /identity/v3/groups/58975584643c4b44a79722ad8c2ba30c => generated 226 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:12.098351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d684a-fbcb-4a13-ada9-545a1545e3bc None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:12.137595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:12.244349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:12.294941 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e9375-126b-4df2-b3dd-2ecae9e548c9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:12.311519 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d684a-fbcb-4a13-ada9-545a1545e3bc None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:12.350799 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9126b-d39f-45a4-9d8b-28bf9fb8bb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:12.352233 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9126b-d39f-45a4-9d8b-28bf9fb8bb36 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:12.352681 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9126b-d39f-45a4-9d8b-28bf9fb8bb36 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:12.353092 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9126b-d39f-45a4-9d8b-28bf9fb8bb36 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:12.354242 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9126b-d39f-45a4-9d8b-28bf9fb8bb36 None admin] RBAC: Adding query filter params (name=member) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:12.354778 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9126b-d39f-45a4-9d8b-28bf9fb8bb36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:12.356514 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9126b-d39f-45a4-9d8b-28bf9fb8bb36 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:12.362242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc554a-59c1-4feb-a8e5-3db0c1aaa753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:12.363504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc554a-59c1-4feb-a8e5-3db0c1aaa753 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:12.363925 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc554a-59c1-4feb-a8e5-3db0c1aaa753 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:12.364328 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc554a-59c1-4feb-a8e5-3db0c1aaa753 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:12.365204 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc554a-59c1-4feb-a8e5-3db0c1aaa753 None admin] RBAC: Adding query filter params (group.id=58975584643c4b44a79722ad8c2ba30c, role.id=d18e4897bf3648498a472a2ea8174f8b, scope.project.id=3f0bb97051994c89b3544850957b48be) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:12.365639 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc554a-59c1-4feb-a8e5-3db0c1aaa753 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:12.370602 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc554a-59c1-4feb-a8e5-3db0c1aaa753 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:12.371777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 178/347] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:35:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:12.374087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693c5c7-f05d-4aee-9bc3-b9aad11a7e46 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:12.374087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693c5c7-f05d-4aee-9bc3-b9aad11a7e46 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:12.374087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7693c5c7-f05d-4aee-9bc3-b9aad11a7e46 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:12.377873 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 179/348] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:12.377873 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693c5c7-f05d-4aee-9bc3-b9aad11a7e46 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:12.383677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 170/349] 10.222.0.33 () {56 vars in 1376 bytes} [Tue Oct 4 12:35:12 2022] GET /identity/v3/role_assignments?group.id=58975584643c4b44a79722ad8c2ba30c&scope.project.id=3f0bb97051994c89b3544850957b48be&role.id=d18e4897bf3648498a472a2ea8174f8b => generated 260 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:12.386375 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805e521-07cf-449e-8477-646d1413af17 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:12.386827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805e521-07cf-449e-8477-646d1413af17 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:12.387297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805e521-07cf-449e-8477-646d1413af17 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:12.391184 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 171/350] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:12.393227 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5b0db-62b5-47fc-9244-7414076e43a2 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:12.393659 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5b0db-62b5-47fc-9244-7414076e43a2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:12.394172 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5b0db-62b5-47fc-9244-7414076e43a2 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:12.396934 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 172/351] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:12.398372 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5b0db-62b5-47fc-9244-7414076e43a2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:12.562610 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:12.582630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7693c5c7-f05d-4aee-9bc3-b9aad11a7e46 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:12.616374 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5b0db-62b5-47fc-9244-7414076e43a2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:12.650346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bb0d4-ae91-443e-9701-f77694766f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:12.651374 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bb0d4-ae91-443e-9701-f77694766f66 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:12.651835 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bb0d4-ae91-443e-9701-f77694766f66 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:12.652268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bb0d4-ae91-443e-9701-f77694766f66 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:12.660618 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:12.669389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141bb0d4-ae91-443e-9701-f77694766f66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-141bb0d4-ae91-443e-9701-f77694766f66 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:12.678229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:12.680002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:12.681606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 180/352] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:35:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:12.683557 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d167e-ec7e-4635-997a-577475199ac8 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:12.683557 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d167e-ec7e-4635-997a-577475199ac8 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:12.683778 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d167e-ec7e-4635-997a-577475199ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:12.704232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecbd9f-9065-4d8f-9324-30d825ff4553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:12.705632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecbd9f-9065-4d8f-9324-30d825ff4553 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:12.706113 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecbd9f-9065-4d8f-9324-30d825ff4553 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:12.706545 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecbd9f-9065-4d8f-9324-30d825ff4553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:12.724271 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-cc1d167e-ec7e-4635-997a-577475199ac8 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:12.725897 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-cc1d167e-ec7e-4635-997a-577475199ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:12.847824 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecbd9f-9065-4d8f-9324-30d825ff4553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:12.850531 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecbd9f-9065-4d8f-9324-30d825ff4553 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:12.854447 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 181/353] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:12.856960 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea728d9d-7a3b-4652-bf37-4f7c06b4a0db None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:12.857274 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea728d9d-7a3b-4652-bf37-4f7c06b4a0db None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:12.857523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea728d9d-7a3b-4652-bf37-4f7c06b4a0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:12.892051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-ea728d9d-7a3b-4652-bf37-4f7c06b4a0db None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:12.895875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-ea728d9d-7a3b-4652-bf37-4f7c06b4a0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:13.015942 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 173/354] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:35:13.017254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa5758-0506-4f3e-b7f4-341531175c4e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:13.017534 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa5758-0506-4f3e-b7f4-341531175c4e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:13.017906 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa5758-0506-4f3e-b7f4-341531175c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:13.040744 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e6aa5758-0506-4f3e-b7f4-341531175c4e None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:13.042631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e6aa5758-0506-4f3e-b7f4-341531175c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:13.045426 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 182/355] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:13.048798 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea728d9d-7a3b-4652-bf37-4f7c06b4a0db None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:13.068025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:13.098683 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:13.197688 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 174/356] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:13.203131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa5758-0506-4f3e-b7f4-341531175c4e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:13.338766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea728d9d-7a3b-4652-bf37-4f7c06b4a0db None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:13.339585 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa5758-0506-4f3e-b7f4-341531175c4e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:13.401224 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662fda5-a327-48be-9755-a62f655e4926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:13.401830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662fda5-a327-48be-9755-a62f655e4926 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:13.402149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662fda5-a327-48be-9755-a62f655e4926 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:13.402404 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662fda5-a327-48be-9755-a62f655e4926 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:13.403710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662fda5-a327-48be-9755-a62f655e4926 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:13.405172 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662fda5-a327-48be-9755-a62f655e4926 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:13.408319 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 175/357] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:13 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:13.409955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662fda5-a327-48be-9755-a62f655e4926 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:13.435508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fc692-461b-4fd8-ae06-6cf814ab2a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:13.436392 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fc692-461b-4fd8-ae06-6cf814ab2a10 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:13.436805 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fc692-461b-4fd8-ae06-6cf814ab2a10 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:13.437203 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fc692-461b-4fd8-ae06-6cf814ab2a10 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:13.438527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171fc692-461b-4fd8-ae06-6cf814ab2a10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:13.438986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171fc692-461b-4fd8-ae06-6cf814ab2a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:13.441141 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171fc692-461b-4fd8-ae06-6cf814ab2a10 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:13.452285 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:13.453110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 183/358] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:13.455419 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fc692-461b-4fd8-ae06-6cf814ab2a10 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:13.540179 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:13.552060 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662fda5-a327-48be-9755-a62f655e4926 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:13.572225 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fc692-461b-4fd8-ae06-6cf814ab2a10 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:13.597727 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23df621-f1d3-40ad-9e8f-0548e6dbadb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:13.598685 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23df621-f1d3-40ad-9e8f-0548e6dbadb4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:13.599145 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23df621-f1d3-40ad-9e8f-0548e6dbadb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:13.599600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23df621-f1d3-40ad-9e8f-0548e6dbadb4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:13.613626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e081ec-1e98-48a5-b03d-bc560485755e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:13.614710 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e081ec-1e98-48a5-b03d-bc560485755e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:13.615227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e081ec-1e98-48a5-b03d-bc560485755e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:13.615336 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23df621-f1d3-40ad-9e8f-0548e6dbadb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:13.615824 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e081ec-1e98-48a5-b03d-bc560485755e None admin] PATH_INFO: `/v3/groups/58975584643c4b44a79722ad8c2ba30c` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:13.617348 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e081ec-1e98-48a5-b03d-bc560485755e None admin] RBAC: Authorizing `identity:get_group(group_id=58975584643c4b44a79722ad8c2ba30c)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:13.619146 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e081ec-1e98-48a5-b03d-bc560485755e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c23df621-f1d3-40ad-9e8f-0548e6dbadb4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:13.620391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:13.621254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:13.622561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:13.622561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:13.622561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:13.622561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:13.622561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:13.622561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:13.622561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:13.622561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:13.622561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:13.622561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:13.622561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:13.622561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:13.624304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 184/359] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:13 2022] GET /identity/v3/groups/58975584643c4b44a79722ad8c2ba30c => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:13.625410 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e081ec-1e98-48a5-b03d-bc560485755e None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:13.626828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 176/360] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:35:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:13.627818 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23df621-f1d3-40ad-9e8f-0548e6dbadb4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:13.728666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:13.746285 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e081ec-1e98-48a5-b03d-bc560485755e None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:13.758372 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23df621-f1d3-40ad-9e8f-0548e6dbadb4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:13.786981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb95f3-e46c-4edc-8b62-f62ed1fc1e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:13.787999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb95f3-e46c-4edc-8b62-f62ed1fc1e20 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:13.788508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb95f3-e46c-4edc-8b62-f62ed1fc1e20 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:13.788976 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb95f3-e46c-4edc-8b62-f62ed1fc1e20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:13.801862 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a24229-357a-4342-9674-3429e5e22a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:13.802488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a24229-357a-4342-9674-3429e5e22a30 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:13.802770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a24229-357a-4342-9674-3429e5e22a30 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:13.803049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a24229-357a-4342-9674-3429e5e22a30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:13.803494 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb95f3-e46c-4edc-8b62-f62ed1fc1e20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-a3cb95f3-e46c-4edc-8b62-f62ed1fc1e20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:13.809530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:13.810718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:13.811547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:13.811547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:13.811547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:13.811547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:13.811547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:13.811547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:13.811547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:13.811547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:13.811547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:13.811547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:13.811547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:35:13.811547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:13.814311 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 185/361] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:35:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:13.815478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb95f3-e46c-4edc-8b62-f62ed1fc1e20 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:13.824327 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:13.892287 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a24229-357a-4342-9674-3429e5e22a30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:13.893800 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a24229-357a-4342-9674-3429e5e22a30 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:13.947265 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb95f3-e46c-4edc-8b62-f62ed1fc1e20 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:13.999614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc28d1a-3ce5-4cad-afb7-b9ebe281f51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:14.000446 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc28d1a-3ce5-4cad-afb7-b9ebe281f51b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:14.000718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc28d1a-3ce5-4cad-afb7-b9ebe281f51b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:14.000992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc28d1a-3ce5-4cad-afb7-b9ebe281f51b None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:14.001698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc28d1a-3ce5-4cad-afb7-b9ebe281f51b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:14.002005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc28d1a-3ce5-4cad-afb7-b9ebe281f51b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:14.003496 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc28d1a-3ce5-4cad-afb7-b9ebe281f51b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:14.021242 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 186/362] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:14.023220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc28d1a-3ce5-4cad-afb7-b9ebe281f51b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:14.030181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:14.064371 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 177/363] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:35:14.065035 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a24229-357a-4342-9674-3429e5e22a30 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:14.132957 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:14.197959 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc28d1a-3ce5-4cad-afb7-b9ebe281f51b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:14.252031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139e5fc-bbb7-482a-9000-cdda15602076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:14.252031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139e5fc-bbb7-482a-9000-cdda15602076 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:14.252031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139e5fc-bbb7-482a-9000-cdda15602076 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:14.252031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139e5fc-bbb7-482a-9000-cdda15602076 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:14.252403 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139e5fc-bbb7-482a-9000-cdda15602076 None admin] RBAC: Adding query filter params (group.id=58975584643c4b44a79722ad8c2ba30c, role.id=d18e4897bf3648498a472a2ea8174f8b, scope.project.id=8f97028a6a564e6b924279df149adb74) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:14.252471 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139e5fc-bbb7-482a-9000-cdda15602076 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:14.253178 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a24229-357a-4342-9674-3429e5e22a30 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:14.254292 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139e5fc-bbb7-482a-9000-cdda15602076 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:14.267421 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 187/364] 10.222.0.33 () {56 vars in 1376 bytes} [Tue Oct 4 12:35:14 2022] GET /identity/v3/role_assignments?group.id=58975584643c4b44a79722ad8c2ba30c&scope.project.id=8f97028a6a564e6b924279df149adb74&role.id=d18e4897bf3648498a472a2ea8174f8b => generated 260 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:14.268388 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139e5fc-bbb7-482a-9000-cdda15602076 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:14.313352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e717d-d9e1-4901-8beb-579d869799b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:14.313991 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e717d-d9e1-4901-8beb-579d869799b9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:14.314373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e717d-d9e1-4901-8beb-579d869799b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:14.314865 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e717d-d9e1-4901-8beb-579d869799b9 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:14.315587 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e717d-d9e1-4901-8beb-579d869799b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:14.315894 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e717d-d9e1-4901-8beb-579d869799b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:14.317354 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e717d-d9e1-4901-8beb-579d869799b9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:14.330089 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 178/365] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:14.330954 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e717d-d9e1-4901-8beb-579d869799b9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:14.388346 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:14.456850 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139e5fc-bbb7-482a-9000-cdda15602076 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:14.478060 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e717d-d9e1-4901-8beb-579d869799b9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:14.510161 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:14.513824 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9030929-5d5d-4def-9f1e-11fcb2c31091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:14.514952 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9030929-5d5d-4def-9f1e-11fcb2c31091 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:14.515438 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9030929-5d5d-4def-9f1e-11fcb2c31091 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:14.515908 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9030929-5d5d-4def-9f1e-11fcb2c31091 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:14.543257 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec20b4-c7fa-4adb-b07d-9091c8d1ba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:14.544224 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec20b4-c7fa-4adb-b07d-9091c8d1ba01 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:14.544688 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec20b4-c7fa-4adb-b07d-9091c8d1ba01 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:14.545137 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec20b4-c7fa-4adb-b07d-9091c8d1ba01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:14.546639 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9030929-5d5d-4def-9f1e-11fcb2c31091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-f9030929-5d5d-4def-9f1e-11fcb2c31091 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:14.555155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:14.556110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:14.556877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:14.556877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:14.556877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:14.556877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:14.556877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:14.556877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:14.556877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:14.556877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:14.556877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:14.556877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:14.556877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 12:35:14.556877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:14.559418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 188/366] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:14.560602 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9030929-5d5d-4def-9f1e-11fcb2c31091 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:14.688687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec20b4-c7fa-4adb-b07d-9091c8d1ba01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:14.690231 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec20b4-c7fa-4adb-b07d-9091c8d1ba01 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:14.761315 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9030929-5d5d-4def-9f1e-11fcb2c31091 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:14.763886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:14.832744 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7e450-2db9-4747-ae01-13963110597b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:14.834910 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7e450-2db9-4747-ae01-13963110597b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:14.835407 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7e450-2db9-4747-ae01-13963110597b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:14.835888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7e450-2db9-4747-ae01-13963110597b None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:14.837966 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7e450-2db9-4747-ae01-13963110597b None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:14.841366 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7e450-2db9-4747-ae01-13963110597b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:14.845975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 189/367] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:14 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:14.847037 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 179/368] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:35:14.847719 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec20b4-c7fa-4adb-b07d-9091c8d1ba01 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:14.851728 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7e450-2db9-4747-ae01-13963110597b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:14.967625 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:14.981814 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec20b4-c7fa-4adb-b07d-9091c8d1ba01 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:15.033072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc199f-ff70-463c-bd3c-8a9de1575015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:15.034386 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc199f-ff70-463c-bd3c-8a9de1575015 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:15.034807 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc199f-ff70-463c-bd3c-8a9de1575015 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:15.035217 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc199f-ff70-463c-bd3c-8a9de1575015 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:15.036043 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc199f-ff70-463c-bd3c-8a9de1575015 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:15.041554 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc199f-ff70-463c-bd3c-8a9de1575015 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:15.041554 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc199f-ff70-463c-bd3c-8a9de1575015 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:15.052262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:15.060008 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7e450-2db9-4747-ae01-13963110597b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:15.060185 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 180/369] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:35:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:15.061670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc199f-ff70-463c-bd3c-8a9de1575015 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:15.103886 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354979a-a93e-444e-8bea-e9de0ed0caf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:15.104949 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354979a-a93e-444e-8bea-e9de0ed0caf5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:15.105435 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354979a-a93e-444e-8bea-e9de0ed0caf5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:15.105900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354979a-a93e-444e-8bea-e9de0ed0caf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:15.186606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354979a-a93e-444e-8bea-e9de0ed0caf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:15.188585 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354979a-a93e-444e-8bea-e9de0ed0caf5 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:15.270268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:15.292725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc199f-ff70-463c-bd3c-8a9de1575015 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:15.362625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14058029-24da-4465-bedf-0acbd537b399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:15.363683 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14058029-24da-4465-bedf-0acbd537b399 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:15.364160 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14058029-24da-4465-bedf-0acbd537b399 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:15.364648 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14058029-24da-4465-bedf-0acbd537b399 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:15.366602 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14058029-24da-4465-bedf-0acbd537b399 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:15.369342 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14058029-24da-4465-bedf-0acbd537b399 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:15.373019 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 181/370] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:15 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:15.374419 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14058029-24da-4465-bedf-0acbd537b399 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:15.377357 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 190/371] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:35:15.379285 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354979a-a93e-444e-8bea-e9de0ed0caf5 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:15.532454 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:15.596201 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354979a-a93e-444e-8bea-e9de0ed0caf5 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:15.603317 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14058029-24da-4465-bedf-0acbd537b399 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:15.655802 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddc75c-797f-447e-9ceb-6862b3b9b23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:15.657071 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddc75c-797f-447e-9ceb-6862b3b9b23e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:15.657697 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddc75c-797f-447e-9ceb-6862b3b9b23e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:15.658346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddc75c-797f-447e-9ceb-6862b3b9b23e None admin] PATH_INFO: `/v3/users/63e16848dd4543a39a644d2605808e38` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:15.660230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddc75c-797f-447e-9ceb-6862b3b9b23e None admin] RBAC: Authorizing `identity:get_user(user_id=63e16848dd4543a39a644d2605808e38)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:15.661938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddc75c-797f-447e-9ceb-6862b3b9b23e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:15.664123 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 191/372] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:35:15 2022] GET /identity/v3/users/63e16848dd4543a39a644d2605808e38 => generated 293 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:15.668081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddc75c-797f-447e-9ceb-6862b3b9b23e None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:15.679148 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82396dc6-9a9d-40d6-9286-a421cddd260c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:15.680945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82396dc6-9a9d-40d6-9286-a421cddd260c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:15.681473 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82396dc6-9a9d-40d6-9286-a421cddd260c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:15.681945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82396dc6-9a9d-40d6-9286-a421cddd260c None admin] PATH_INFO: `/v3/groups/58975584643c4b44a79722ad8c2ba30c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:15.687012 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82396dc6-9a9d-40d6-9286-a421cddd260c None admin] RBAC: Authorizing `identity:get_group(group_id=58975584643c4b44a79722ad8c2ba30c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:15.688055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82396dc6-9a9d-40d6-9286-a421cddd260c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:15.694649 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 182/373] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:15 2022] GET /identity/v3/groups/58975584643c4b44a79722ad8c2ba30c => generated 226 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:15.695352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82396dc6-9a9d-40d6-9286-a421cddd260c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:15.704686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:15.827468 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82396dc6-9a9d-40d6-9286-a421cddd260c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:15.839896 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddc75c-797f-447e-9ceb-6862b3b9b23e None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:15.871050 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd2bfd-57b0-41f3-ac8c-a1ea85633338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:15.871661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd2bfd-57b0-41f3-ac8c-a1ea85633338 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:15.871995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd2bfd-57b0-41f3-ac8c-a1ea85633338 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:15.872274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd2bfd-57b0-41f3-ac8c-a1ea85633338 None admin] PATH_INFO: `/v3/projects/f85b0de3065741e781a91cbb56d099f5` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:15.873527 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd2bfd-57b0-41f3-ac8c-a1ea85633338 None admin] RBAC: Authorizing `identity:get_project(project_id=f85b0de3065741e781a91cbb56d099f5)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:15.875092 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd2bfd-57b0-41f3-ac8c-a1ea85633338 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:15.877268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 183/374] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:15 2022] GET /identity/v3/projects/f85b0de3065741e781a91cbb56d099f5 => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:15.878145 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd2bfd-57b0-41f3-ac8c-a1ea85633338 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:15.881976 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:15.913655 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8a6ba-e2c8-40d7-b2a0-1b2f995c642c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:15.914774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8a6ba-e2c8-40d7-b2a0-1b2f995c642c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:15.915404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8a6ba-e2c8-40d7-b2a0-1b2f995c642c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:15.915895 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8a6ba-e2c8-40d7-b2a0-1b2f995c642c None admin] PATH_INFO: `/v3/groups/58975584643c4b44a79722ad8c2ba30c` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:15.918273 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8a6ba-e2c8-40d7-b2a0-1b2f995c642c None admin] RBAC: Authorizing `identity:get_group(group_id=58975584643c4b44a79722ad8c2ba30c)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:15.921160 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada8a6ba-e2c8-40d7-b2a0-1b2f995c642c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:15.924798 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 192/375] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:15 2022] GET /identity/v3/groups/58975584643c4b44a79722ad8c2ba30c => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:15.926379 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8a6ba-e2c8-40d7-b2a0-1b2f995c642c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:15.998802 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd2bfd-57b0-41f3-ac8c-a1ea85633338 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:16.040431 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e9851-789c-4aac-91db-bf076f3c19dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:16.041440 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e9851-789c-4aac-91db-bf076f3c19dc None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:16.041915 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e9851-789c-4aac-91db-bf076f3c19dc None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:16.043025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e9851-789c-4aac-91db-bf076f3c19dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:16.044336 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e9851-789c-4aac-91db-bf076f3c19dc None admin] RBAC: Adding query filter params (role.id=1b0fe0772e714d1ba454d992ef8fa266, user.id=63e16848dd4543a39a644d2605808e38, scope.system=all) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:16.044890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e9851-789c-4aac-91db-bf076f3c19dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:16.047645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e9851-789c-4aac-91db-bf076f3c19dc None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:16.078064 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 184/376] 10.222.0.33 () {56 vars in 1308 bytes} [Tue Oct 4 12:35:15 2022] GET /identity/v3/role_assignments?user.id=63e16848dd4543a39a644d2605808e38&scope.system=all&role.id=1b0fe0772e714d1ba454d992ef8fa266 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:16.078064 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e9851-789c-4aac-91db-bf076f3c19dc None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:16.083707 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:16.096898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8a6ba-e2c8-40d7-b2a0-1b2f995c642c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:16.140463 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f34ee-70c0-4ac6-aac4-f73c42316ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:16.141106 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f34ee-70c0-4ac6-aac4-f73c42316ef9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:16.141449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f34ee-70c0-4ac6-aac4-f73c42316ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:16.141821 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f34ee-70c0-4ac6-aac4-f73c42316ef9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:16.142668 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f34ee-70c0-4ac6-aac4-f73c42316ef9 None admin] RBAC: Adding query filter params (group.id=58975584643c4b44a79722ad8c2ba30c, role.id=ca13aabb28b6442f8540fa4e8bb1b35a, scope.project.id=3f0bb97051994c89b3544850957b48be) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:16.143046 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f34ee-70c0-4ac6-aac4-f73c42316ef9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:16.144657 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f34ee-70c0-4ac6-aac4-f73c42316ef9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:16.151481 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 193/377] 10.222.0.33 () {56 vars in 1376 bytes} [Tue Oct 4 12:35:15 2022] GET /identity/v3/role_assignments?group.id=58975584643c4b44a79722ad8c2ba30c&scope.project.id=3f0bb97051994c89b3544850957b48be&role.id=ca13aabb28b6442f8540fa4e8bb1b35a => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:16.152364 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f34ee-70c0-4ac6-aac4-f73c42316ef9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:16.221307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:16.302388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e9851-789c-4aac-91db-bf076f3c19dc None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:16.352636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a54e9c-8161-412d-8460-c0eda1f06cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:16.352636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a54e9c-8161-412d-8460-c0eda1f06cf8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:16.352960 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a54e9c-8161-412d-8460-c0eda1f06cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:16.353168 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a54e9c-8161-412d-8460-c0eda1f06cf8 None admin] PATH_INFO: `/v3/groups/3c6433a6da4d46ce8cc4ce660efcc8f9` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:16.362796 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a54e9c-8161-412d-8460-c0eda1f06cf8 None admin] RBAC: Authorizing `identity:get_group(group_id=3c6433a6da4d46ce8cc4ce660efcc8f9)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:16.364322 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a54e9c-8161-412d-8460-c0eda1f06cf8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:16.367349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 185/378] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:16 2022] GET /identity/v3/groups/3c6433a6da4d46ce8cc4ce660efcc8f9 => generated 229 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:16.368089 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d633fc2-8ff6-488f-adb2-27e786255982 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:16.368483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d633fc2-8ff6-488f-adb2-27e786255982 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:16.368761 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d633fc2-8ff6-488f-adb2-27e786255982 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:16.371052 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 186/379] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:16.377598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b693b31-a0d2-4849-9972-0fa1cfd8bb4d None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:16.377910 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b693b31-a0d2-4849-9972-0fa1cfd8bb4d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:16.378187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b693b31-a0d2-4849-9972-0fa1cfd8bb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:16.391153 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f34ee-70c0-4ac6-aac4-f73c42316ef9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:16.405964 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-7b693b31-a0d2-4849-9972-0fa1cfd8bb4d None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:16.406494 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7b693b31-a0d2-4849-9972-0fa1cfd8bb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:16.448505 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fe4bf-04a2-4e4f-a0f6-e80193dc87af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:16.450279 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fe4bf-04a2-4e4f-a0f6-e80193dc87af None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:16.451071 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fe4bf-04a2-4e4f-a0f6-e80193dc87af None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:16.451570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fe4bf-04a2-4e4f-a0f6-e80193dc87af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:16.452841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fe4bf-04a2-4e4f-a0f6-e80193dc87af None admin] RBAC: Adding query filter params (group.id=58975584643c4b44a79722ad8c2ba30c, role.id=ca13aabb28b6442f8540fa4e8bb1b35a, scope.project.id=8f97028a6a564e6b924279df149adb74) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:16.453409 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fe4bf-04a2-4e4f-a0f6-e80193dc87af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:16.455132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fe4bf-04a2-4e4f-a0f6-e80193dc87af None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:16.467485 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 194/380] 10.222.0.33 () {56 vars in 1376 bytes} [Tue Oct 4 12:35:16 2022] GET /identity/v3/role_assignments?group.id=58975584643c4b44a79722ad8c2ba30c&scope.project.id=8f97028a6a564e6b924279df149adb74&role.id=ca13aabb28b6442f8540fa4e8bb1b35a => generated 260 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:16.469604 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fe4bf-04a2-4e4f-a0f6-e80193dc87af None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:16.518618 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 187/381] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:35:16.532513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:16.636265 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fe4bf-04a2-4e4f-a0f6-e80193dc87af None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:16.683584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c38111-bcac-4ccb-91a2-be5796e9547e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:16.684594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c38111-bcac-4ccb-91a2-be5796e9547e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:16.685093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c38111-bcac-4ccb-91a2-be5796e9547e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:16.685562 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c38111-bcac-4ccb-91a2-be5796e9547e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:16.686566 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c38111-bcac-4ccb-91a2-be5796e9547e None admin] RBAC: Adding query filter params (group.id=3c6433a6da4d46ce8cc4ce660efcc8f9, role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=f85b0de3065741e781a91cbb56d099f5) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:16.687104 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c38111-bcac-4ccb-91a2-be5796e9547e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:16.688870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c38111-bcac-4ccb-91a2-be5796e9547e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:16.701854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 195/382] 10.222.0.33 () {56 vars in 1376 bytes} [Tue Oct 4 12:35:16 2022] GET /identity/v3/role_assignments?group.id=3c6433a6da4d46ce8cc4ce660efcc8f9&scope.project.id=f85b0de3065741e781a91cbb56d099f5&role.id=fc68d072459d4720b9666c93bd535d63 => generated 260 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:17.061963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b693b31-a0d2-4849-9972-0fa1cfd8bb4d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:17.153541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:17.254521 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b693b31-a0d2-4849-9972-0fa1cfd8bb4d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:17.332995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34736b-a862-4351-9c28-345a5db46ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:17.334025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34736b-a862-4351-9c28-345a5db46ed5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:17.334552 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34736b-a862-4351-9c28-345a5db46ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:17.335036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34736b-a862-4351-9c28-345a5db46ed5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:17.355594 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34736b-a862-4351-9c28-345a5db46ed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0d34736b-a862-4351-9c28-345a5db46ed5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:17.369106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:17.370677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:17.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:17.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:17.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:17.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:17.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:17.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:17.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:17.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:17.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:17.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:17.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:17.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:17.372583 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 188/383] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:17.382229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c38111-bcac-4ccb-91a2-be5796e9547e None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:17.564730 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:17.591015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c38111-bcac-4ccb-91a2-be5796e9547e None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:17.634310 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98eed7-9ee7-4427-be6f-5075e4e1d2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:17.634934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98eed7-9ee7-4427-be6f-5075e4e1d2de None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:17.635214 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98eed7-9ee7-4427-be6f-5075e4e1d2de None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:17.635482 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98eed7-9ee7-4427-be6f-5075e4e1d2de None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:17.636184 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98eed7-9ee7-4427-be6f-5075e4e1d2de None admin] RBAC: Adding query filter params (name=member) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:17.636482 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98eed7-9ee7-4427-be6f-5075e4e1d2de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:17.637880 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98eed7-9ee7-4427-be6f-5075e4e1d2de None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:17.651171 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 196/384] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:35:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:17.656657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34736b-a862-4351-9c28-345a5db46ed5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:17.903618 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34736b-a862-4351-9c28-345a5db46ed5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:17.934697 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c604357-4bf1-4108-a6d8-b8954e94bcc8 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:17.934953 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c604357-4bf1-4108-a6d8-b8954e94bcc8 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:17.935279 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c604357-4bf1-4108-a6d8-b8954e94bcc8 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:17.937099 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 197/385] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:17.941092 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:17.945150 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d0cdd-e43e-4225-8c4c-8a52c3593f22 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:17.945550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d0cdd-e43e-4225-8c4c-8a52c3593f22 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:17.946723 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d0cdd-e43e-4225-8c4c-8a52c3593f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:18.000113 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00252638-fd57-45d6-8b76-7f19c6760596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:18.001144 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00252638-fd57-45d6-8b76-7f19c6760596 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:18.001612 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00252638-fd57-45d6-8b76-7f19c6760596 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:18.002117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00252638-fd57-45d6-8b76-7f19c6760596 None admin] PATH_INFO: `/v3/groups/58975584643c4b44a79722ad8c2ba30c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:18.004463 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-e10d0cdd-e43e-4225-8c4c-8a52c3593f22 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:18.006434 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-e10d0cdd-e43e-4225-8c4c-8a52c3593f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:18.007938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00252638-fd57-45d6-8b76-7f19c6760596 None admin] RBAC: Authorizing `identity:get_group(group_id=58975584643c4b44a79722ad8c2ba30c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:18.010546 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00252638-fd57-45d6-8b76-7f19c6760596 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:18.020243 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 189/386] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:17 2022] GET /identity/v3/groups/58975584643c4b44a79722ad8c2ba30c => generated 226 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:18.021772 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00252638-fd57-45d6-8b76-7f19c6760596 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:18.207967 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 198/387] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:35:18.239866 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00252638-fd57-45d6-8b76-7f19c6760596 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:18.308187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab809ffb-d68e-4be1-98a7-0ce382e42e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:18.309333 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab809ffb-d68e-4be1-98a7-0ce382e42e27 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:18.309823 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab809ffb-d68e-4be1-98a7-0ce382e42e27 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:18.310325 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab809ffb-d68e-4be1-98a7-0ce382e42e27 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:18.312323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab809ffb-d68e-4be1-98a7-0ce382e42e27 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:18.315044 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab809ffb-d68e-4be1-98a7-0ce382e42e27 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:18.318678 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 190/388] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:18 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:18.327509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d0cdd-e43e-4225-8c4c-8a52c3593f22 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:18.478823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:18.511493 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d0cdd-e43e-4225-8c4c-8a52c3593f22 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:18.516770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab809ffb-d68e-4be1-98a7-0ce382e42e27 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:18.579347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260e09d-71a7-47a4-967e-1f7a14ef9f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:18.580038 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260e09d-71a7-47a4-967e-1f7a14ef9f82 None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:18.580315 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260e09d-71a7-47a4-967e-1f7a14ef9f82 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:18.580588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260e09d-71a7-47a4-967e-1f7a14ef9f82 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be/groups/58975584643c4b44a79722ad8c2ba30c/roles/d18e4897bf3648498a472a2ea8174f8b` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:18.583942 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260e09d-71a7-47a4-967e-1f7a14ef9f82 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f0bb97051994c89b3544850957b48be, group_id=58975584643c4b44a79722ad8c2ba30c, role_id=d18e4897bf3648498a472a2ea8174f8b)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:18.585424 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260e09d-71a7-47a4-967e-1f7a14ef9f82 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:18.614142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 199/389] 10.222.0.33 () {58 vars in 1442 bytes} [Tue Oct 4 12:35:18 2022] PUT /identity/v3/projects/3f0bb97051994c89b3544850957b48be/groups/58975584643c4b44a79722ad8c2ba30c/roles/d18e4897bf3648498a472a2ea8174f8b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:35:18.626544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:18.717585 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab809ffb-d68e-4be1-98a7-0ce382e42e27 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:18.780047 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efff5a-1a74-4dde-8cef-6ca05e68a348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:18.781165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efff5a-1a74-4dde-8cef-6ca05e68a348 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:18.781774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efff5a-1a74-4dde-8cef-6ca05e68a348 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:18.782331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efff5a-1a74-4dde-8cef-6ca05e68a348 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:18.800791 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72efff5a-1a74-4dde-8cef-6ca05e68a348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-72efff5a-1a74-4dde-8cef-6ca05e68a348 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:18.807310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:18.808792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:18.809553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:18.809553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:18.809553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:18.809553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:18.809553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:18.809553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:18.809553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:18.809553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:18.809553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:18.809553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:18.809553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:18.809553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:18.810174 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 191/390] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:18.816748 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260e09d-71a7-47a4-967e-1f7a14ef9f82 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:18.982073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260e09d-71a7-47a4-967e-1f7a14ef9f82 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:19.031821 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5139c5-e65a-4cc5-a28b-9a65ea0e7a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:19.032364 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5139c5-e65a-4cc5-a28b-9a65ea0e7a7d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:19.032643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5139c5-e65a-4cc5-a28b-9a65ea0e7a7d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:19.032936 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5139c5-e65a-4cc5-a28b-9a65ea0e7a7d None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:19.033581 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5139c5-e65a-4cc5-a28b-9a65ea0e7a7d None admin] RBAC: Adding query filter params (name=member) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:19.033940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5139c5-e65a-4cc5-a28b-9a65ea0e7a7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:19.035531 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5139c5-e65a-4cc5-a28b-9a65ea0e7a7d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:19.038266 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:19.051677 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 200/391] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:35:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:19.057756 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efff5a-1a74-4dde-8cef-6ca05e68a348 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:19.176541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efff5a-1a74-4dde-8cef-6ca05e68a348 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:19.191240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:19.196128 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892916b-f575-45b1-9260-065539b009c7 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:19.196411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892916b-f575-45b1-9260-065539b009c7 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:19.196683 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892916b-f575-45b1-9260-065539b009c7 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:19.199448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 201/392] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:35:19.222532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1096ae-bf70-48e9-bec1-994f3d371871 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:19.222941 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1096ae-bf70-48e9-bec1-994f3d371871 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:19.223095 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1096ae-bf70-48e9-bec1-994f3d371871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:19.236682 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a60bc8-76a8-43e5-8873-b99ae5f31049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:19.238281 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a60bc8-76a8-43e5-8873-b99ae5f31049 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:19.239081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a60bc8-76a8-43e5-8873-b99ae5f31049 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:19.239899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a60bc8-76a8-43e5-8873-b99ae5f31049 None admin] PATH_INFO: `/v3/groups/58975584643c4b44a79722ad8c2ba30c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:19.243273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a60bc8-76a8-43e5-8873-b99ae5f31049 None admin] RBAC: Authorizing `identity:get_group(group_id=58975584643c4b44a79722ad8c2ba30c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:19.246399 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a60bc8-76a8-43e5-8873-b99ae5f31049 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:19.251515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 192/393] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:19 2022] GET /identity/v3/groups/58975584643c4b44a79722ad8c2ba30c => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:19.261895 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a60bc8-76a8-43e5-8873-b99ae5f31049 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:19.287310 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-5c1096ae-bf70-48e9-bec1-994f3d371871 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:19.288454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-5c1096ae-bf70-48e9-bec1-994f3d371871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:19.423363 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 202/394] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:19.435457 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a60bc8-76a8-43e5-8873-b99ae5f31049 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:19.494934 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ae5d9-df78-4d6e-8eaf-90824c6d9207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:19.496513 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ae5d9-df78-4d6e-8eaf-90824c6d9207 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:19.496995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ae5d9-df78-4d6e-8eaf-90824c6d9207 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:19.497494 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ae5d9-df78-4d6e-8eaf-90824c6d9207 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:19.499082 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ae5d9-df78-4d6e-8eaf-90824c6d9207 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:19.500793 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ae5d9-df78-4d6e-8eaf-90824c6d9207 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:19.502786 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 193/395] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:19 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:19.508922 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1096ae-bf70-48e9-bec1-994f3d371871 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:19.590996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965f635-62bb-4efb-b32b-388089433c33 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:19.591165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965f635-62bb-4efb-b32b-388089433c33 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:19.591424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965f635-62bb-4efb-b32b-388089433c33 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:19.593015 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 194/396] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:35:19.600519 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a2f95-3040-4a9a-ac39-879f42570697 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:19.600724 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a2f95-3040-4a9a-ac39-879f42570697 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:19.600921 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a2f95-3040-4a9a-ac39-879f42570697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:19.624538 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1096ae-bf70-48e9-bec1-994f3d371871 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:19.634010 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-332a2f95-3040-4a9a-ac39-879f42570697 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:19.635175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-332a2f95-3040-4a9a-ac39-879f42570697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:19.658385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:19.670151 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cdfad-8c2b-4c7b-8209-7cf018feb27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:19.671910 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cdfad-8c2b-4c7b-8209-7cf018feb27a None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:19.672420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cdfad-8c2b-4c7b-8209-7cf018feb27a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:19.672914 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888cdfad-8c2b-4c7b-8209-7cf018feb27a None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74/groups/58975584643c4b44a79722ad8c2ba30c/roles/d18e4897bf3648498a472a2ea8174f8b` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:19.684031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888cdfad-8c2b-4c7b-8209-7cf018feb27a None admin] RBAC: Authorizing `identity:create_grant(project_id=8f97028a6a564e6b924279df149adb74, group_id=58975584643c4b44a79722ad8c2ba30c, role_id=d18e4897bf3648498a472a2ea8174f8b)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:19.686975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888cdfad-8c2b-4c7b-8209-7cf018feb27a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:19.709796 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:19.721266 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 203/397] 10.222.0.33 () {58 vars in 1442 bytes} [Tue Oct 4 12:35:19 2022] PUT /identity/v3/projects/8f97028a6a564e6b924279df149adb74/groups/58975584643c4b44a79722ad8c2ba30c/roles/d18e4897bf3648498a472a2ea8174f8b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:35:19.770778 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 195/398] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:35:19.779656 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cdfad-8c2b-4c7b-8209-7cf018feb27a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:19.934822 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888cdfad-8c2b-4c7b-8209-7cf018feb27a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:19.985496 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcb1e3-b1d7-4829-8eef-561980fe7e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:19.986188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcb1e3-b1d7-4829-8eef-561980fe7e68 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:19.986465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcb1e3-b1d7-4829-8eef-561980fe7e68 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:19.986734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcb1e3-b1d7-4829-8eef-561980fe7e68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:20.000091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbcb1e3-b1d7-4829-8eef-561980fe7e68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-8fbcb1e3-b1d7-4829-8eef-561980fe7e68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:20.005530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:20.006448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:20.007230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 204/399] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:35:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:20.017378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a2f95-3040-4a9a-ac39-879f42570697 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:20.163257 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a2f95-3040-4a9a-ac39-879f42570697 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:20.215846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f4446-6f17-4388-9422-d838e2af56e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:20.216759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f4446-6f17-4388-9422-d838e2af56e2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:20.217213 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f4446-6f17-4388-9422-d838e2af56e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:20.217678 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f4446-6f17-4388-9422-d838e2af56e2 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:20.218551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f4446-6f17-4388-9422-d838e2af56e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:20.219031 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f4446-6f17-4388-9422-d838e2af56e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:20.220597 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f4446-6f17-4388-9422-d838e2af56e2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:20.234055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 196/400] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:20.240476 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcb1e3-b1d7-4829-8eef-561980fe7e68 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:20.329140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:20.357566 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f4446-6f17-4388-9422-d838e2af56e2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:20.420757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:20.426352 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcb1e3-b1d7-4829-8eef-561980fe7e68 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:20.475446 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc4523-e6ea-4ece-8fae-13f34775e46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:20.476561 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc4523-e6ea-4ece-8fae-13f34775e46b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:20.477072 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc4523-e6ea-4ece-8fae-13f34775e46b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:20.477576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc4523-e6ea-4ece-8fae-13f34775e46b None admin] PATH_INFO: `/v3/groups/58975584643c4b44a79722ad8c2ba30c` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:20.481335 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc4523-e6ea-4ece-8fae-13f34775e46b None admin] RBAC: Authorizing `identity:get_group(group_id=58975584643c4b44a79722ad8c2ba30c)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:20.485251 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc4523-e6ea-4ece-8fae-13f34775e46b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:20.495828 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 205/401] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:20 2022] GET /identity/v3/groups/58975584643c4b44a79722ad8c2ba30c => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:20.495828 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f835aba-d2a8-4a21-bde7-c3f50cae3d31 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:20.495828 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f835aba-d2a8-4a21-bde7-c3f50cae3d31 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:20.495828 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f835aba-d2a8-4a21-bde7-c3f50cae3d31 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:20.496719 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 206/402] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:20.500703 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f835aba-d2a8-4a21-bde7-c3f50cae3d31 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:20.530606 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f4446-6f17-4388-9422-d838e2af56e2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:20.573524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8db3ba-517b-4f8f-9984-6ef5a24bbc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:20.574231 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8db3ba-517b-4f8f-9984-6ef5a24bbc3c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:20.574509 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8db3ba-517b-4f8f-9984-6ef5a24bbc3c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:20.574785 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8db3ba-517b-4f8f-9984-6ef5a24bbc3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:20.587883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8db3ba-517b-4f8f-9984-6ef5a24bbc3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-ef8db3ba-517b-4f8f-9984-6ef5a24bbc3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:20.593952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:20.595220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:20.596006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 197/403] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:35:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:20.596921 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cfe6c-14f5-4085-acb1-5b2e9cd93ad4 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:20.596989 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cfe6c-14f5-4085-acb1-5b2e9cd93ad4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:20.602466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cfe6c-14f5-4085-acb1-5b2e9cd93ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:20.626705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-dc2cfe6c-14f5-4085-acb1-5b2e9cd93ad4 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:20.628420 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dc2cfe6c-14f5-4085-acb1-5b2e9cd93ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:20.655413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f835aba-d2a8-4a21-bde7-c3f50cae3d31 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:20.675774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:20.697541 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b0dbd-169d-43de-b364-d7f9a1eeeb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:20.698513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b0dbd-169d-43de-b364-d7f9a1eeeb1c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:20.698937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b0dbd-169d-43de-b364-d7f9a1eeeb1c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:20.699382 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b0dbd-169d-43de-b364-d7f9a1eeeb1c None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:20.701320 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b0dbd-169d-43de-b364-d7f9a1eeeb1c None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:20.702988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b0dbd-169d-43de-b364-d7f9a1eeeb1c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:20.704925 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 207/404] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:20 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:20.705992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b0dbd-169d-43de-b364-d7f9a1eeeb1c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:20.765541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 198/405] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:20.775375 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cfe6c-14f5-4085-acb1-5b2e9cd93ad4 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:20.829827 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b0dbd-169d-43de-b364-d7f9a1eeeb1c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:20.866062 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:20.937412 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73e9a2-99a8-4d23-9df0-ed2a2fcea6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:20.938948 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73e9a2-99a8-4d23-9df0-ed2a2fcea6eb None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:20.939718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73e9a2-99a8-4d23-9df0-ed2a2fcea6eb None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:20.940310 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73e9a2-99a8-4d23-9df0-ed2a2fcea6eb None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:20.941779 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73e9a2-99a8-4d23-9df0-ed2a2fcea6eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:20.944745 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73e9a2-99a8-4d23-9df0-ed2a2fcea6eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:20.947807 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73e9a2-99a8-4d23-9df0-ed2a2fcea6eb None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:20.984287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 208/406] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:21.000279 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73e9a2-99a8-4d23-9df0-ed2a2fcea6eb None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:21.095305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cfe6c-14f5-4085-acb1-5b2e9cd93ad4 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:21.156449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:21.194840 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5bd83-6cac-49ba-9c9b-99795635054c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:21.196025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5bd83-6cac-49ba-9c9b-99795635054c None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:21.196526 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5bd83-6cac-49ba-9c9b-99795635054c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:21.197011 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5bd83-6cac-49ba-9c9b-99795635054c None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be/groups/58975584643c4b44a79722ad8c2ba30c/roles/ca13aabb28b6442f8540fa4e8bb1b35a` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:21.212957 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5bd83-6cac-49ba-9c9b-99795635054c None admin] RBAC: Authorizing `identity:create_grant(project_id=3f0bb97051994c89b3544850957b48be, group_id=58975584643c4b44a79722ad8c2ba30c, role_id=ca13aabb28b6442f8540fa4e8bb1b35a)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:21.216517 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5bd83-6cac-49ba-9c9b-99795635054c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:21.244953 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 199/407] 10.222.0.33 () {58 vars in 1442 bytes} [Tue Oct 4 12:35:20 2022] PUT /identity/v3/projects/3f0bb97051994c89b3544850957b48be/groups/58975584643c4b44a79722ad8c2ba30c/roles/ca13aabb28b6442f8540fa4e8bb1b35a => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:35:21.247392 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5bd83-6cac-49ba-9c9b-99795635054c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:21.332706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73e9a2-99a8-4d23-9df0-ed2a2fcea6eb None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:21.401112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:21.412605 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c901c80-c3b0-4483-8a82-e1edc1883106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:21.414991 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c901c80-c3b0-4483-8a82-e1edc1883106 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:21.415519 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c901c80-c3b0-4483-8a82-e1edc1883106 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:21.415970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c901c80-c3b0-4483-8a82-e1edc1883106 None admin] PATH_INFO: `/v3/groups/58975584643c4b44a79722ad8c2ba30c` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:21.418990 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c901c80-c3b0-4483-8a82-e1edc1883106 None admin] RBAC: Authorizing `identity:get_group(group_id=58975584643c4b44a79722ad8c2ba30c)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:21.422236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c901c80-c3b0-4483-8a82-e1edc1883106 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:21.424315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5bd83-6cac-49ba-9c9b-99795635054c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:21.425835 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 209/408] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:20 2022] GET /identity/v3/groups/58975584643c4b44a79722ad8c2ba30c => generated 226 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:21.428505 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefdbe9-3d51-44e6-b535-065de8bd38e7 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:21.429000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefdbe9-3d51-44e6-b535-065de8bd38e7 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:21.429459 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefdbe9-3d51-44e6-b535-065de8bd38e7 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:21.431992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 210/409] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:21.436918 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefdbe9-3d51-44e6-b535-065de8bd38e7 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:21.504721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb06daf-8325-4c4b-8fd6-6de352129184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:21.505627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb06daf-8325-4c4b-8fd6-6de352129184 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:21.506144 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb06daf-8325-4c4b-8fd6-6de352129184 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:21.506599 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb06daf-8325-4c4b-8fd6-6de352129184 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:21.528582 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb06daf-8325-4c4b-8fd6-6de352129184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-efb06daf-8325-4c4b-8fd6-6de352129184 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:21.539474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:21.541104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:21.542670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 200/410] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:35:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:21.545456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ceb7-a31e-439e-ac5d-d828cc1fb8f9 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:21.545456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ceb7-a31e-439e-ac5d-d828cc1fb8f9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:21.545896 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ceb7-a31e-439e-ac5d-d828cc1fb8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:21.576970 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1d24ceb7-a31e-439e-ac5d-d828cc1fb8f9 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:21.579075 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1d24ceb7-a31e-439e-ac5d-d828cc1fb8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:21.602243 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefdbe9-3d51-44e6-b535-065de8bd38e7 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:21.628530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:21.663882 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e74ae-65b3-4502-ad2e-00a4ea3d3526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:21.664708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e74ae-65b3-4502-ad2e-00a4ea3d3526 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:21.665128 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e74ae-65b3-4502-ad2e-00a4ea3d3526 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:21.665539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e74ae-65b3-4502-ad2e-00a4ea3d3526 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:21.667043 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 201/411] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:21.667217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e74ae-65b3-4502-ad2e-00a4ea3d3526 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:21.667850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24ceb7-a31e-439e-ac5d-d828cc1fb8f9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:21.669188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e74ae-65b3-4502-ad2e-00a4ea3d3526 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:21.672331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 211/412] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:21 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:21.678196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e74ae-65b3-4502-ad2e-00a4ea3d3526 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:21.856328 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:21.902800 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24ceb7-a31e-439e-ac5d-d828cc1fb8f9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:21.936484 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e74ae-65b3-4502-ad2e-00a4ea3d3526 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:21.985513 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bff14f-ad00-40a3-96f7-db6a6d2a492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:21.986664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bff14f-ad00-40a3-96f7-db6a6d2a492a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:21.987185 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bff14f-ad00-40a3-96f7-db6a6d2a492a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:21.987631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bff14f-ad00-40a3-96f7-db6a6d2a492a None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:21.988994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bff14f-ad00-40a3-96f7-db6a6d2a492a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:21.989698 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bff14f-ad00-40a3-96f7-db6a6d2a492a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:21.992294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bff14f-ad00-40a3-96f7-db6a6d2a492a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:21.994547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102d532-73ea-4685-8306-81e89a91b265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:21.995798 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102d532-73ea-4685-8306-81e89a91b265 None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:21.996080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102d532-73ea-4685-8306-81e89a91b265 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:21.996353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102d532-73ea-4685-8306-81e89a91b265 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74/groups/58975584643c4b44a79722ad8c2ba30c/roles/ca13aabb28b6442f8540fa4e8bb1b35a` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:21.998687 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102d532-73ea-4685-8306-81e89a91b265 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f97028a6a564e6b924279df149adb74, group_id=58975584643c4b44a79722ad8c2ba30c, role_id=ca13aabb28b6442f8540fa4e8bb1b35a)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:22.000174 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102d532-73ea-4685-8306-81e89a91b265 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:22.018268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 202/414] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:22.019576 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bff14f-ad00-40a3-96f7-db6a6d2a492a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:22.022155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 212/414] 10.222.0.33 () {58 vars in 1442 bytes} [Tue Oct 4 12:35:21 2022] PUT /identity/v3/projects/8f97028a6a564e6b924279df149adb74/groups/58975584643c4b44a79722ad8c2ba30c/roles/ca13aabb28b6442f8540fa4e8bb1b35a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:35:22.023943 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102d532-73ea-4685-8306-81e89a91b265 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:22.132166 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:22.258611 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bff14f-ad00-40a3-96f7-db6a6d2a492a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:22.259552 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102d532-73ea-4685-8306-81e89a91b265 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:22.308809 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e40881-acc1-419c-af5d-8c24c9d34f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:22.310028 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40881-acc1-419c-af5d-8c24c9d34f51 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:22.310484 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40881-acc1-419c-af5d-8c24c9d34f51 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:22.310904 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40881-acc1-419c-af5d-8c24c9d34f51 None admin] PATH_INFO: `/v3/groups/3c6433a6da4d46ce8cc4ce660efcc8f9` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:22.313182 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e40881-acc1-419c-af5d-8c24c9d34f51 None admin] RBAC: Authorizing `identity:get_group(group_id=3c6433a6da4d46ce8cc4ce660efcc8f9)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:22.315201 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e40881-acc1-419c-af5d-8c24c9d34f51 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:22.318359 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 213/415] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:22 2022] GET /identity/v3/groups/3c6433a6da4d46ce8cc4ce660efcc8f9 => generated 229 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:22.327007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e40881-acc1-419c-af5d-8c24c9d34f51 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:22.339512 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304a20c-42c6-4165-abf2-1659723da0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:22.340568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304a20c-42c6-4165-abf2-1659723da0f5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:22.341589 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304a20c-42c6-4165-abf2-1659723da0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:22.341728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304a20c-42c6-4165-abf2-1659723da0f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:22.367004 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304a20c-42c6-4165-abf2-1659723da0f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-1304a20c-42c6-4165-abf2-1659723da0f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:22.385628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:22.386741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:22.387630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:22.387630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:22.387630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:22.387630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:22.387630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:22.387630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:22.387630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:22.387630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:22.387630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:22.387630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:22.387630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:22.387630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:22.388405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 203/416] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:22.400458 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304a20c-42c6-4165-abf2-1659723da0f5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:22.407478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:22.503343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e40881-acc1-419c-af5d-8c24c9d34f51 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:22.524203 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:22.573834 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828c078-adef-43dd-bee2-1840c38ed8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:22.574967 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828c078-adef-43dd-bee2-1840c38ed8a5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:22.575457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828c078-adef-43dd-bee2-1840c38ed8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:22.575926 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828c078-adef-43dd-bee2-1840c38ed8a5 None admin] PATH_INFO: `/v3/projects/f85b0de3065741e781a91cbb56d099f5` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:22.578238 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828c078-adef-43dd-bee2-1840c38ed8a5 None admin] RBAC: Authorizing `identity:get_project(project_id=f85b0de3065741e781a91cbb56d099f5)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:22.581102 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828c078-adef-43dd-bee2-1840c38ed8a5 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:22.590228 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 214/417] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:22 2022] GET /identity/v3/projects/f85b0de3065741e781a91cbb56d099f5 => generated 344 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:22.592687 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f63078-413c-49b5-a57d-9407e81fba21 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:22.593149 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f63078-413c-49b5-a57d-9407e81fba21 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:22.593614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f63078-413c-49b5-a57d-9407e81fba21 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:22.599934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 215/418] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:22.601528 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f63078-413c-49b5-a57d-9407e81fba21 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:22.631877 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304a20c-42c6-4165-abf2-1659723da0f5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:22.715248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd6462-ae0b-48e4-9173-727e574f19a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:22.716330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd6462-ae0b-48e4-9173-727e574f19a7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:22.717797 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd6462-ae0b-48e4-9173-727e574f19a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:22.718305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd6462-ae0b-48e4-9173-727e574f19a7 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:22.719787 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd6462-ae0b-48e4-9173-727e574f19a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:22.720472 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd6462-ae0b-48e4-9173-727e574f19a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:22.723334 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd6462-ae0b-48e4-9173-727e574f19a7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:22.743266 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 204/419] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:35:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:22.745988 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8c09f-9afc-494e-b036-f918652eb77d None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:22.746589 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8c09f-9afc-494e-b036-f918652eb77d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:22.747059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8c09f-9afc-494e-b036-f918652eb77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:22.784338 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5cb8c09f-9afc-494e-b036-f918652eb77d None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:22.786115 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5cb8c09f-9afc-494e-b036-f918652eb77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:22.811810 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f63078-413c-49b5-a57d-9407e81fba21 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:22.853691 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0b41b-7a12-4152-8b58-0a10088e8864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:22.854812 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0b41b-7a12-4152-8b58-0a10088e8864 None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:22.855393 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0b41b-7a12-4152-8b58-0a10088e8864 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:22.855932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0b41b-7a12-4152-8b58-0a10088e8864 None admin] PATH_INFO: `/v3/projects/f85b0de3065741e781a91cbb56d099f5/groups/3c6433a6da4d46ce8cc4ce660efcc8f9/roles/fc68d072459d4720b9666c93bd535d63` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:22.859027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0b41b-7a12-4152-8b58-0a10088e8864 None admin] RBAC: Authorizing `identity:create_grant(project_id=f85b0de3065741e781a91cbb56d099f5, group_id=3c6433a6da4d46ce8cc4ce660efcc8f9, role_id=fc68d072459d4720b9666c93bd535d63)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:22.861041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0b41b-7a12-4152-8b58-0a10088e8864 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:22.880122 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 216/420] 10.222.0.33 () {58 vars in 1442 bytes} [Tue Oct 4 12:35:22 2022] PUT /identity/v3/projects/f85b0de3065741e781a91cbb56d099f5/groups/3c6433a6da4d46ce8cc4ce660efcc8f9/roles/fc68d072459d4720b9666c93bd535d63 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:35:22.882651 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0b41b-7a12-4152-8b58-0a10088e8864 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:22.918442 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:22.932496 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 205/421] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:22.979007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:23.123170 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0b41b-7a12-4152-8b58-0a10088e8864 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:23.204007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5f848-0dd3-4cb3-be5c-5ccf28d89e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:23.205009 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5f848-0dd3-4cb3-be5c-5ccf28d89e87 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:23.205561 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5f848-0dd3-4cb3-be5c-5ccf28d89e87 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:23.206074 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5f848-0dd3-4cb3-be5c-5ccf28d89e87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:23.328150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8c09f-9afc-494e-b036-f918652eb77d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:23.341365 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5f848-0dd3-4cb3-be5c-5ccf28d89e87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:23.344179 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5f848-0dd3-4cb3-be5c-5ccf28d89e87 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:23.419556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:23.447414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8c09f-9afc-494e-b036-f918652eb77d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:23.488795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170e9bc-62e9-4f76-b62d-de8a975b141f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:23.489446 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170e9bc-62e9-4f76-b62d-de8a975b141f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:23.489726 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170e9bc-62e9-4f76-b62d-de8a975b141f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:23.490009 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170e9bc-62e9-4f76-b62d-de8a975b141f None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:23.502343 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170e9bc-62e9-4f76-b62d-de8a975b141f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a170e9bc-62e9-4f76-b62d-de8a975b141f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:23.507294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:23.509014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:23.510617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:23.510617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:23.510617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:23.510617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:23.510617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:23.510617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:23.510617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:23.510617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:23.510617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:23.510617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:23.510617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 12:35:23.510617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:23.514093 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 206/422] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:23.515315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170e9bc-62e9-4f76-b62d-de8a975b141f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:23.573099 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 217/423] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:35:23.579695 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:23.582122 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5f848-0dd3-4cb3-be5c-5ccf28d89e87 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:23.633674 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170e9bc-62e9-4f76-b62d-de8a975b141f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:23.674595 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f92ced-5c6d-45ef-a521-a7d1d4ffc1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:23.675593 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f92ced-5c6d-45ef-a521-a7d1d4ffc1bb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:23.676075 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f92ced-5c6d-45ef-a521-a7d1d4ffc1bb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:23.676544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f92ced-5c6d-45ef-a521-a7d1d4ffc1bb None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:23.677478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f92ced-5c6d-45ef-a521-a7d1d4ffc1bb None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:23.677987 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f92ced-5c6d-45ef-a521-a7d1d4ffc1bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:23.679743 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f92ced-5c6d-45ef-a521-a7d1d4ffc1bb None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:23.694199 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 207/424] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:35:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:23.697106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f92ced-5c6d-45ef-a521-a7d1d4ffc1bb None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:23.820298 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f92ced-5c6d-45ef-a521-a7d1d4ffc1bb None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:23.840483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:23.865233 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d1abd-345a-4c4f-8bc1-8738014bab68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:23.866279 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d1abd-345a-4c4f-8bc1-8738014bab68 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:23.866811 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d1abd-345a-4c4f-8bc1-8738014bab68 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:23.867283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d1abd-345a-4c4f-8bc1-8738014bab68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:23.904443 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5f848-0dd3-4cb3-be5c-5ccf28d89e87 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:23.946105 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d1abd-345a-4c4f-8bc1-8738014bab68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:23.947925 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d1abd-345a-4c4f-8bc1-8738014bab68 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:24.019285 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac7434-bc12-4480-b036-8f28b682b444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:24.020686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac7434-bc12-4480-b036-8f28b682b444 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:24.021181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac7434-bc12-4480-b036-8f28b682b444 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:24.021670 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac7434-bc12-4480-b036-8f28b682b444 None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:24.023863 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac7434-bc12-4480-b036-8f28b682b444 None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:24.027254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac7434-bc12-4480-b036-8f28b682b444 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:24.039224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 218/425] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:23 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:24.048526 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac7434-bc12-4480-b036-8f28b682b444 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:24.070192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 208/426] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 12:35:24.072251 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d1abd-345a-4c4f-8bc1-8738014bab68 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:24.074415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:24.098744 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:24.216915 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d1abd-345a-4c4f-8bc1-8738014bab68 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:24.298440 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac7434-bc12-4480-b036-8f28b682b444 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:24.306969 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d5514-bddb-4d19-b739-dc2193fab8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:24.307681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d5514-bddb-4d19-b739-dc2193fab8aa None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:24.307965 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d5514-bddb-4d19-b739-dc2193fab8aa None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:24.308244 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d5514-bddb-4d19-b739-dc2193fab8aa None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:24.309531 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d5514-bddb-4d19-b739-dc2193fab8aa None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:24.311166 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d5514-bddb-4d19-b739-dc2193fab8aa None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:24.317443 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 209/427] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:24 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:24.319562 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d5514-bddb-4d19-b739-dc2193fab8aa None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:24.364889 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dad408-ef39-4b88-b2fc-a64c20eab806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:24.365880 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dad408-ef39-4b88-b2fc-a64c20eab806 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:24.366358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dad408-ef39-4b88-b2fc-a64c20eab806 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:24.366823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dad408-ef39-4b88-b2fc-a64c20eab806 None admin] PATH_INFO: `/v3/groups/58975584643c4b44a79722ad8c2ba30c` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:24.371177 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dad408-ef39-4b88-b2fc-a64c20eab806 None admin] RBAC: Authorizing `identity:get_group(group_id=58975584643c4b44a79722ad8c2ba30c)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:24.372981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dad408-ef39-4b88-b2fc-a64c20eab806 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:24.378884 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 219/428] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:24 2022] GET /identity/v3/groups/58975584643c4b44a79722ad8c2ba30c => generated 226 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:24.384970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dad408-ef39-4b88-b2fc-a64c20eab806 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:24.499302 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:24.521834 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:24.544779 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d5514-bddb-4d19-b739-dc2193fab8aa None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:24.554762 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dad408-ef39-4b88-b2fc-a64c20eab806 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:24.610634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba9f4b-7bea-4182-9973-2fafa4a2ab27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:24.611323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba9f4b-7bea-4182-9973-2fafa4a2ab27 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:24.611598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba9f4b-7bea-4182-9973-2fafa4a2ab27 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:24.611876 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba9f4b-7bea-4182-9973-2fafa4a2ab27 None admin] PATH_INFO: `/v3/groups/58975584643c4b44a79722ad8c2ba30c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:24.616308 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba9f4b-7bea-4182-9973-2fafa4a2ab27 None admin] RBAC: Authorizing `identity:get_group(group_id=58975584643c4b44a79722ad8c2ba30c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:24.617997 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba9f4b-7bea-4182-9973-2fafa4a2ab27 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:24.620690 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 210/429] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:24 2022] GET /identity/v3/groups/58975584643c4b44a79722ad8c2ba30c => generated 226 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:24.626877 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba9f4b-7bea-4182-9973-2fafa4a2ab27 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:24.627055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183e995-8f46-4dde-a447-8785c2d17e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:24.628295 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183e995-8f46-4dde-a447-8785c2d17e6f None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:24.628797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183e995-8f46-4dde-a447-8785c2d17e6f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:24.629345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183e995-8f46-4dde-a447-8785c2d17e6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:24.630717 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183e995-8f46-4dde-a447-8785c2d17e6f None admin] RBAC: Adding query filter params (group.id=58975584643c4b44a79722ad8c2ba30c, role.id=d18e4897bf3648498a472a2ea8174f8b, scope.project.id=3f0bb97051994c89b3544850957b48be) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:24.631273 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183e995-8f46-4dde-a447-8785c2d17e6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:24.634017 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183e995-8f46-4dde-a447-8785c2d17e6f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:24.650356 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 220/430] 10.222.0.33 () {56 vars in 1376 bytes} [Tue Oct 4 12:35:24 2022] GET /identity/v3/role_assignments?group.id=58975584643c4b44a79722ad8c2ba30c&scope.project.id=3f0bb97051994c89b3544850957b48be&role.id=d18e4897bf3648498a472a2ea8174f8b => generated 611 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:24.783168 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba9f4b-7bea-4182-9973-2fafa4a2ab27 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:24.845470 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa67d03d-6eab-4ffa-bf32-1e3e61581477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:24.846581 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67d03d-6eab-4ffa-bf32-1e3e61581477 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:24.847095 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67d03d-6eab-4ffa-bf32-1e3e61581477 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:24.847600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa67d03d-6eab-4ffa-bf32-1e3e61581477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:24.848581 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67d03d-6eab-4ffa-bf32-1e3e61581477 None admin] RBAC: Adding query filter params (group.id=58975584643c4b44a79722ad8c2ba30c, role.id=d18e4897bf3648498a472a2ea8174f8b, scope.project.id=8f97028a6a564e6b924279df149adb74) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:24.849115 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67d03d-6eab-4ffa-bf32-1e3e61581477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:24.850880 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa67d03d-6eab-4ffa-bf32-1e3e61581477 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:24.863705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 211/431] 10.222.0.33 () {56 vars in 1376 bytes} [Tue Oct 4 12:35:24 2022] GET /identity/v3/role_assignments?group.id=58975584643c4b44a79722ad8c2ba30c&scope.project.id=8f97028a6a564e6b924279df149adb74&role.id=d18e4897bf3648498a472a2ea8174f8b => generated 611 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:25.096496 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a28b5-ff88-4155-b178-ee154a5e736a None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:25.096999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a28b5-ff88-4155-b178-ee154a5e736a None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:25.097361 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a28b5-ff88-4155-b178-ee154a5e736a None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:25.100403 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 221/432] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:25.109517 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df4b34-b3b7-4cbe-8b6c-c02796dfcb27 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:25.109931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df4b34-b3b7-4cbe-8b6c-c02796dfcb27 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:25.110266 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df4b34-b3b7-4cbe-8b6c-c02796dfcb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:25.112875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc6875-464c-4151-8d69-c7111d17f104 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:25.113942 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc6875-464c-4151-8d69-c7111d17f104 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:25.114796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc6875-464c-4151-8d69-c7111d17f104 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:25.118242 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 222/433] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:25.126970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2406ba-6252-45bd-82e2-a7f786c532c8 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:25.128113 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2406ba-6252-45bd-82e2-a7f786c532c8 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:25.128883 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2406ba-6252-45bd-82e2-a7f786c532c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:25.145717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-02df4b34-b3b7-4cbe-8b6c-c02796dfcb27 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:25.147849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-02df4b34-b3b7-4cbe-8b6c-c02796dfcb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:25.165582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-eb2406ba-6252-45bd-82e2-a7f786c532c8 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:25.167720 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-eb2406ba-6252-45bd-82e2-a7f786c532c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:25.174509 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:25.297488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 212/434] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:25.308614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 223/435] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:35:25.580436 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df4b34-b3b7-4cbe-8b6c-c02796dfcb27 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:25.713351 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2406ba-6252-45bd-82e2-a7f786c532c8 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:25.757860 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df4b34-b3b7-4cbe-8b6c-c02796dfcb27 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:25.808544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39c674-59db-4ce8-b392-c4e18b36938e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:25.809339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39c674-59db-4ce8-b392-c4e18b36938e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:25.809803 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39c674-59db-4ce8-b392-c4e18b36938e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:25.810262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39c674-59db-4ce8-b392-c4e18b36938e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:25.828935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39c674-59db-4ce8-b392-c4e18b36938e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0b39c674-59db-4ce8-b392-c4e18b36938e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:25.837737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:25.839933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:25.841625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 213/436] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:35:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:25.846600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39c674-59db-4ce8-b392-c4e18b36938e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:25.862326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:25.864935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:25.926349 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2406ba-6252-45bd-82e2-a7f786c532c8 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:25.983773 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc75783-f80a-451d-af50-70306f95ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:25.984615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc75783-f80a-451d-af50-70306f95ce1d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:25.985027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc75783-f80a-451d-af50-70306f95ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:25.985438 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc75783-f80a-451d-af50-70306f95ce1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:26.005280 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc75783-f80a-451d-af50-70306f95ce1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-dcc75783-f80a-451d-af50-70306f95ce1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:26.014808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:26.015971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:26.017041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 224/437] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:35:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:26.022004 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39c674-59db-4ce8-b392-c4e18b36938e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:26.026558 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc75783-f80a-451d-af50-70306f95ce1d None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:26.078256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349b8e4-e152-4acd-88e8-6159deadcc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:26.079392 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349b8e4-e152-4acd-88e8-6159deadcc5d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:26.079572 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349b8e4-e152-4acd-88e8-6159deadcc5d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:26.079869 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349b8e4-e152-4acd-88e8-6159deadcc5d None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:26.081055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349b8e4-e152-4acd-88e8-6159deadcc5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:26.082011 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349b8e4-e152-4acd-88e8-6159deadcc5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:26.083708 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349b8e4-e152-4acd-88e8-6159deadcc5d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:26.101467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 214/438] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:26.112591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349b8e4-e152-4acd-88e8-6159deadcc5d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:26.205070 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc75783-f80a-451d-af50-70306f95ce1d None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:26.230771 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:26.247226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:26.265260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97d97a-cc89-44e0-86d6-ae12710622bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:26.266066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97d97a-cc89-44e0-86d6-ae12710622bb None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:26.266505 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97d97a-cc89-44e0-86d6-ae12710622bb None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:26.266871 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97d97a-cc89-44e0-86d6-ae12710622bb None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:26.267674 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97d97a-cc89-44e0-86d6-ae12710622bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:26.268073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97d97a-cc89-44e0-86d6-ae12710622bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:26.269959 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97d97a-cc89-44e0-86d6-ae12710622bb None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:26.277684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349b8e4-e152-4acd-88e8-6159deadcc5d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:26.280354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 225/439] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:26.281853 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb2b39-9965-465f-8037-4c5b864021b0 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:26.282266 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb2b39-9965-465f-8037-4c5b864021b0 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:26.282702 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb2b39-9965-465f-8037-4c5b864021b0 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:26.284389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 226/440] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:26.289685 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb2b39-9965-465f-8037-4c5b864021b0 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:26.330841 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b220c1-6eb8-4520-8e75-20c6645183b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:26.331685 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b220c1-6eb8-4520-8e75-20c6645183b2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:26.332026 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b220c1-6eb8-4520-8e75-20c6645183b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:26.332360 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b220c1-6eb8-4520-8e75-20c6645183b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:26.420076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb2b39-9965-465f-8037-4c5b864021b0 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:26.425974 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b220c1-6eb8-4520-8e75-20c6645183b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:26.427626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b220c1-6eb8-4520-8e75-20c6645183b2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:26.480362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5327f9-7f57-4dc6-b920-e4c91e4631e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:26.481292 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5327f9-7f57-4dc6-b920-e4c91e4631e8 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:26.481787 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5327f9-7f57-4dc6-b920-e4c91e4631e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:26.482308 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5327f9-7f57-4dc6-b920-e4c91e4631e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:26.499441 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:26.543370 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 215/441] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 12:35:26.545963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f1908-7c76-4815-af7c-ee6e17eed89b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:26.546469 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f1908-7c76-4815-af7c-ee6e17eed89b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:26.546793 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f1908-7c76-4815-af7c-ee6e17eed89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:26.566926 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:26.575431 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b02f1908-7c76-4815-af7c-ee6e17eed89b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:26.577355 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b02f1908-7c76-4815-af7c-ee6e17eed89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:26.584104 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5327f9-7f57-4dc6-b920-e4c91e4631e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:26.585768 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5327f9-7f57-4dc6-b920-e4c91e4631e8 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:26.678603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 216/442] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:26.679643 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f1908-7c76-4815-af7c-ee6e17eed89b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:26.761920 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 227/443] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 12:35:26.768079 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5327f9-7f57-4dc6-b920-e4c91e4631e8 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:26.807593 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:26.813896 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f1908-7c76-4815-af7c-ee6e17eed89b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:26.879158 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec79a97-4764-41a7-acd2-b969e4989a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:26.879930 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec79a97-4764-41a7-acd2-b969e4989a18 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:26.880205 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec79a97-4764-41a7-acd2-b969e4989a18 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:26.880480 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec79a97-4764-41a7-acd2-b969e4989a18 None admin] PATH_INFO: `/v3/projects/8f97028a6a564e6b924279df149adb74` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:26.881935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec79a97-4764-41a7-acd2-b969e4989a18 None admin] RBAC: Authorizing `identity:get_project(project_id=8f97028a6a564e6b924279df149adb74)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:26.883670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec79a97-4764-41a7-acd2-b969e4989a18 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:26.886437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 217/444] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:26 2022] GET /identity/v3/projects/8f97028a6a564e6b924279df149adb74 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:26.898716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec79a97-4764-41a7-acd2-b969e4989a18 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:26.900200 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:26.913061 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5327f9-7f57-4dc6-b920-e4c91e4631e8 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:26.951165 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba91e98-643c-4a92-b4c8-a481d63e241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:26.951686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba91e98-643c-4a92-b4c8-a481d63e241a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:26.951964 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba91e98-643c-4a92-b4c8-a481d63e241a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:26.952240 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba91e98-643c-4a92-b4c8-a481d63e241a None admin] PATH_INFO: `/v3/projects/3f0bb97051994c89b3544850957b48be` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:26.953411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba91e98-643c-4a92-b4c8-a481d63e241a None admin] RBAC: Authorizing `identity:get_project(project_id=3f0bb97051994c89b3544850957b48be)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:26.954921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba91e98-643c-4a92-b4c8-a481d63e241a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:26.956576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 228/445] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:26 2022] GET /identity/v3/projects/3f0bb97051994c89b3544850957b48be => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:26.957348 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba91e98-643c-4a92-b4c8-a481d63e241a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:27.025936 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec79a97-4764-41a7-acd2-b969e4989a18 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:27.063305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b7d96-25c5-45d0-872f-8b29725eef01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:27.063948 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b7d96-25c5-45d0-872f-8b29725eef01 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:27.064222 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b7d96-25c5-45d0-872f-8b29725eef01 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:27.064491 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b7d96-25c5-45d0-872f-8b29725eef01 None admin] PATH_INFO: `/v3/groups/58975584643c4b44a79722ad8c2ba30c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:27.065642 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b7d96-25c5-45d0-872f-8b29725eef01 None admin] RBAC: Authorizing `identity:get_group(group_id=58975584643c4b44a79722ad8c2ba30c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:27.067135 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b7d96-25c5-45d0-872f-8b29725eef01 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:27.068714 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 218/446] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:26 2022] GET /identity/v3/groups/58975584643c4b44a79722ad8c2ba30c => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:27.069476 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b7d96-25c5-45d0-872f-8b29725eef01 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:27.101326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba91e98-643c-4a92-b4c8-a481d63e241a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:27.106293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:27.143322 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d3d90-e81d-4f18-8ad4-dfac002ef9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:27.144436 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d3d90-e81d-4f18-8ad4-dfac002ef9f0 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:27.145202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d3d90-e81d-4f18-8ad4-dfac002ef9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:27.145686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d3d90-e81d-4f18-8ad4-dfac002ef9f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:27.164529 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d3d90-e81d-4f18-8ad4-dfac002ef9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-964d3d90-e81d-4f18-8ad4-dfac002ef9f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:27.170031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:27.171157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 229/447] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:35:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:27.171981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d3d90-e81d-4f18-8ad4-dfac002ef9f0 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:27.183397 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b7d96-25c5-45d0-872f-8b29725eef01 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:27.191550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:27.224142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a153bb-b302-4953-a11e-f38a376f2cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:27.224764 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a153bb-b302-4953-a11e-f38a376f2cbc None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:27.225335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a153bb-b302-4953-a11e-f38a376f2cbc None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:27.225335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a153bb-b302-4953-a11e-f38a376f2cbc None admin] PATH_INFO: `/v3/groups/58975584643c4b44a79722ad8c2ba30c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:27.226576 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a153bb-b302-4953-a11e-f38a376f2cbc None admin] RBAC: Authorizing `identity:get_group(group_id=58975584643c4b44a79722ad8c2ba30c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:27.228318 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a153bb-b302-4953-a11e-f38a376f2cbc None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:27.230536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 219/448] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:27 2022] GET /identity/v3/groups/58975584643c4b44a79722ad8c2ba30c => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:27.231325 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a153bb-b302-4953-a11e-f38a376f2cbc None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:27.292474 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d3d90-e81d-4f18-8ad4-dfac002ef9f0 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:27.330236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65775a-75dc-4777-b923-7f0d32a2f15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:27.330840 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65775a-75dc-4777-b923-7f0d32a2f15b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:27.331120 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65775a-75dc-4777-b923-7f0d32a2f15b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:27.331398 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65775a-75dc-4777-b923-7f0d32a2f15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:27.332133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65775a-75dc-4777-b923-7f0d32a2f15b None admin] RBAC: Adding query filter params (group.id=58975584643c4b44a79722ad8c2ba30c, role.id=ca13aabb28b6442f8540fa4e8bb1b35a, scope.project.id=8f97028a6a564e6b924279df149adb74) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:27.332434 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65775a-75dc-4777-b923-7f0d32a2f15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:27.333898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65775a-75dc-4777-b923-7f0d32a2f15b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:27.340694 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 230/449] 10.222.0.33 () {56 vars in 1376 bytes} [Tue Oct 4 12:35:27 2022] GET /identity/v3/role_assignments?group.id=58975584643c4b44a79722ad8c2ba30c&scope.project.id=8f97028a6a564e6b924279df149adb74&role.id=ca13aabb28b6442f8540fa4e8bb1b35a => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:27.341504 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65775a-75dc-4777-b923-7f0d32a2f15b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:27.358240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a153bb-b302-4953-a11e-f38a376f2cbc None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:27.360588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:27.406828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f500748-0104-44bb-8648-534cf7e5ce70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:27.407474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f500748-0104-44bb-8648-534cf7e5ce70 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:27.407747 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f500748-0104-44bb-8648-534cf7e5ce70 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:27.408015 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f500748-0104-44bb-8648-534cf7e5ce70 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:27.408843 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f500748-0104-44bb-8648-534cf7e5ce70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:27.409152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f500748-0104-44bb-8648-534cf7e5ce70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:27.410622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f500748-0104-44bb-8648-534cf7e5ce70 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:27.425985 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 220/450] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:27.427185 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f500748-0104-44bb-8648-534cf7e5ce70 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:27.466075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:27.490102 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65775a-75dc-4777-b923-7f0d32a2f15b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:27.548787 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cbe8e-741e-4f97-b086-1bfff7d2695c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:27.549796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cbe8e-741e-4f97-b086-1bfff7d2695c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:27.550634 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cbe8e-741e-4f97-b086-1bfff7d2695c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:27.550821 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cbe8e-741e-4f97-b086-1bfff7d2695c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:27.554618 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cbe8e-741e-4f97-b086-1bfff7d2695c None admin] RBAC: Adding query filter params (group.id=58975584643c4b44a79722ad8c2ba30c, role.id=ca13aabb28b6442f8540fa4e8bb1b35a, scope.project.id=3f0bb97051994c89b3544850957b48be) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:27.554618 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cbe8e-741e-4f97-b086-1bfff7d2695c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:27.555227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cbe8e-741e-4f97-b086-1bfff7d2695c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:27.559711 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f500748-0104-44bb-8648-534cf7e5ce70 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:27.568565 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 231/451] 10.222.0.33 () {56 vars in 1376 bytes} [Tue Oct 4 12:35:27 2022] GET /identity/v3/role_assignments?group.id=58975584643c4b44a79722ad8c2ba30c&scope.project.id=3f0bb97051994c89b3544850957b48be&role.id=ca13aabb28b6442f8540fa4e8bb1b35a => generated 611 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:27.601710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cf09c-4d09-4ca9-ac76-e5f53ae8893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:27.602439 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cf09c-4d09-4ca9-ac76-e5f53ae8893a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:27.602719 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cf09c-4d09-4ca9-ac76-e5f53ae8893a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:27.602993 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cf09c-4d09-4ca9-ac76-e5f53ae8893a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:27.652895 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:27.697822 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cf09c-4d09-4ca9-ac76-e5f53ae8893a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:27.699405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cf09c-4d09-4ca9-ac76-e5f53ae8893a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:27.812349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 221/452] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 12:35:27.818386 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cbe8e-741e-4f97-b086-1bfff7d2695c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:27.933616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cbe8e-741e-4f97-b086-1bfff7d2695c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:27.972925 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d76faa-13c5-495c-8b55-9a2295167769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:27.973556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d76faa-13c5-495c-8b55-9a2295167769 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:27.973831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d76faa-13c5-495c-8b55-9a2295167769 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:27.974149 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d76faa-13c5-495c-8b55-9a2295167769 None admin] PATH_INFO: `/v3/projects/f85b0de3065741e781a91cbb56d099f5` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:27.975304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d76faa-13c5-495c-8b55-9a2295167769 None admin] RBAC: Authorizing `identity:get_project(project_id=f85b0de3065741e781a91cbb56d099f5)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:27.976969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d76faa-13c5-495c-8b55-9a2295167769 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:27.978448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 232/453] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:35:27 2022] GET /identity/v3/projects/f85b0de3065741e781a91cbb56d099f5 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:27.984907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cf09c-4d09-4ca9-ac76-e5f53ae8893a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:28.065718 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:28.108673 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cf09c-4d09-4ca9-ac76-e5f53ae8893a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:28.152523 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedcb11-cab6-455a-a8b2-92fe4483b30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:28.153163 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedcb11-cab6-455a-a8b2-92fe4483b30b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:28.153450 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedcb11-cab6-455a-a8b2-92fe4483b30b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:28.153721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedcb11-cab6-455a-a8b2-92fe4483b30b None admin] PATH_INFO: `/v3/groups/3c6433a6da4d46ce8cc4ce660efcc8f9` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:28.154914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedcb11-cab6-455a-a8b2-92fe4483b30b None admin] RBAC: Authorizing `identity:get_group(group_id=3c6433a6da4d46ce8cc4ce660efcc8f9)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:28.156388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedcb11-cab6-455a-a8b2-92fe4483b30b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:28.158084 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 222/454] 10.222.0.33 () {56 vars in 1180 bytes} [Tue Oct 4 12:35:27 2022] GET /identity/v3/groups/3c6433a6da4d46ce8cc4ce660efcc8f9 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:28.164228 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d76faa-13c5-495c-8b55-9a2295167769 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:28.182131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:28.290683 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d76faa-13c5-495c-8b55-9a2295167769 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:28.331133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922caa5e-d791-4fa7-bd41-3723275fc3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:28.331873 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922caa5e-d791-4fa7-bd41-3723275fc3a6 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:28.331996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922caa5e-d791-4fa7-bd41-3723275fc3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:28.332198 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922caa5e-d791-4fa7-bd41-3723275fc3a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:28.332951 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922caa5e-d791-4fa7-bd41-3723275fc3a6 None admin] RBAC: Adding query filter params (group.id=3c6433a6da4d46ce8cc4ce660efcc8f9, role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=f85b0de3065741e781a91cbb56d099f5) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:28.337878 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922caa5e-d791-4fa7-bd41-3723275fc3a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:28.337878 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922caa5e-d791-4fa7-bd41-3723275fc3a6 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:28.342689 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 233/455] 10.222.0.33 () {56 vars in 1376 bytes} [Tue Oct 4 12:35:28 2022] GET /identity/v3/role_assignments?group.id=3c6433a6da4d46ce8cc4ce660efcc8f9&scope.project.id=f85b0de3065741e781a91cbb56d099f5&role.id=fc68d072459d4720b9666c93bd535d63 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:31.184594 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297c6e5-ef5d-4bb2-a3f6-133cb58713a2 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:31.185666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297c6e5-ef5d-4bb2-a3f6-133cb58713a2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:31.186192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297c6e5-ef5d-4bb2-a3f6-133cb58713a2 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:31.188141 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 223/456] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:31.194553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06a33e-dfd7-4598-a03e-3b8082a903eb None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:31.195563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06a33e-dfd7-4598-a03e-3b8082a903eb None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:31.196048 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06a33e-dfd7-4598-a03e-3b8082a903eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:31.218891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-bb06a33e-dfd7-4598-a03e-3b8082a903eb None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:31.220468 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-bb06a33e-dfd7-4598-a03e-3b8082a903eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:31.344692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 234/457] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:35:31.625910 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac0a03-4b7b-4460-9faa-af7788abdf70 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:31.626683 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac0a03-4b7b-4460-9faa-af7788abdf70 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:31.627165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac0a03-4b7b-4460-9faa-af7788abdf70 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:31.629162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 224/458] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:31.634863 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d9c91-0011-4302-9bf0-15ee5e2ade5f None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:31.636170 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d9c91-0011-4302-9bf0-15ee5e2ade5f None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:31.636710 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d9c91-0011-4302-9bf0-15ee5e2ade5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:31.659684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-709d9c91-0011-4302-9bf0-15ee5e2ade5f None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:31.661278 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-709d9c91-0011-4302-9bf0-15ee5e2ade5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:31.663831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:31.756848 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 235/459] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:31.801025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac0a03-4b7b-4460-9faa-af7788abdf70 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:31.954068 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac0a03-4b7b-4460-9faa-af7788abdf70 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:31.995920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:32.003839 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7380421-fd36-4487-bf5e-4b3878a3c927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:32.006154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7380421-fd36-4487-bf5e-4b3878a3c927 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:32.006154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7380421-fd36-4487-bf5e-4b3878a3c927 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:32.006154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7380421-fd36-4487-bf5e-4b3878a3c927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:32.018388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7380421-fd36-4487-bf5e-4b3878a3c927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:32.020171 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7380421-fd36-4487-bf5e-4b3878a3c927 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-b7380421-fd36-4487-bf5e-4b3878a3c927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:32.040938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:32.042716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:32.043775 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:32.043775 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:32.043775 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:32.045529 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 225/460] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:32.047555 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd18fb5-53e8-4104-a3ea-0f8fd42c19fe None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:32.047998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd18fb5-53e8-4104-a3ea-0f8fd42c19fe None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:32.048421 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd18fb5-53e8-4104-a3ea-0f8fd42c19fe None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:32.050273 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 236/461] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:32.051414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7380421-fd36-4487-bf5e-4b3878a3c927 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:32.058913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd18fb5-53e8-4104-a3ea-0f8fd42c19fe None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:32.180708 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7380421-fd36-4487-bf5e-4b3878a3c927 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:32.201824 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd18fb5-53e8-4104-a3ea-0f8fd42c19fe None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:32.227417 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781b447-5cd6-426e-a02c-9c0a2213813a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:32.228096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781b447-5cd6-426e-a02c-9c0a2213813a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:32.228674 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781b447-5cd6-426e-a02c-9c0a2213813a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:32.229772 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781b447-5cd6-426e-a02c-9c0a2213813a None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:32.230583 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781b447-5cd6-426e-a02c-9c0a2213813a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:32.230993 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781b447-5cd6-426e-a02c-9c0a2213813a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:32.233493 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781b447-5cd6-426e-a02c-9c0a2213813a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:32.244025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f2473-d6c2-4325-b8d4-1f6dac2ee25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:32.245098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f2473-d6c2-4325-b8d4-1f6dac2ee25a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:32.245599 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f2473-d6c2-4325-b8d4-1f6dac2ee25a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:32.246158 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f2473-d6c2-4325-b8d4-1f6dac2ee25a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:32.251373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 226/462] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:32.255330 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:32.256749 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d55c0-d6b8-4ab1-9158-046b9d59624a None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:32.257044 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d55c0-d6b8-4ab1-9158-046b9d59624a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:32.257441 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d55c0-d6b8-4ab1-9158-046b9d59624a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:32.262174 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f2473-d6c2-4325-b8d4-1f6dac2ee25a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:32.264002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f2473-d6c2-4325-b8d4-1f6dac2ee25a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-f35f2473-d6c2-4325-b8d4-1f6dac2ee25a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:32.283107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:32.284044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:32.284883 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:32.284883 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:32.284883 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:32.286661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-462d55c0-d6b8-4ab1-9158-046b9d59624a None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:32.287354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 237/463] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:32.287912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-462d55c0-d6b8-4ab1-9158-046b9d59624a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:32.288569 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f2473-d6c2-4325-b8d4-1f6dac2ee25a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:32.355483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:32.402137 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 227/464] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:32.403423 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d55c0-d6b8-4ab1-9158-046b9d59624a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:32.409740 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f2473-d6c2-4325-b8d4-1f6dac2ee25a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:32.492628 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a9998-f0fd-49ea-b005-6d3dba89817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:32.493630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a9998-f0fd-49ea-b005-6d3dba89817f None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:32.494155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a9998-f0fd-49ea-b005-6d3dba89817f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:32.494626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a9998-f0fd-49ea-b005-6d3dba89817f None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:32.496740 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a9998-f0fd-49ea-b005-6d3dba89817f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:32.500663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a9998-f0fd-49ea-b005-6d3dba89817f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:32.552158 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 238/465] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:35:32 2022] POST /identity/v3/users => generated 248 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:35:32.602881 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d55c0-d6b8-4ab1-9158-046b9d59624a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:32.659659 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04b7b4-2c82-450a-a69c-d31cb4f7ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:32.660400 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04b7b4-2c82-450a-a69c-d31cb4f7ad29 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:32.660687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04b7b4-2c82-450a-a69c-d31cb4f7ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:32.660964 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04b7b4-2c82-450a-a69c-d31cb4f7ad29 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:32.661693 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04b7b4-2c82-450a-a69c-d31cb4f7ad29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:32.662006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04b7b4-2c82-450a-a69c-d31cb4f7ad29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:32.663632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04b7b4-2c82-450a-a69c-d31cb4f7ad29 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:32.684543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 228/466] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:32.691531 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a9998-f0fd-49ea-b005-6d3dba89817f None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:32.738400 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04b7b4-2c82-450a-a69c-d31cb4f7ad29 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:32.801099 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:32.830531 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:32.897304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a9998-f0fd-49ea-b005-6d3dba89817f None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:32.922965 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04b7b4-2c82-450a-a69c-d31cb4f7ad29 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:32.968344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392f94c-e77f-4e93-a8a7-f4a3e77c8a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:32.969333 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392f94c-e77f-4e93-a8a7-f4a3e77c8a20 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:32.969818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392f94c-e77f-4e93-a8a7-f4a3e77c8a20 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:32.970333 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392f94c-e77f-4e93-a8a7-f4a3e77c8a20 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:32.971461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392f94c-e77f-4e93-a8a7-f4a3e77c8a20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:32.973299 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4cf6ab-d88f-41aa-8784-2d91364c4474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:32.974067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392f94c-e77f-4e93-a8a7-f4a3e77c8a20 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:32.974286 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cf6ab-d88f-41aa-8784-2d91364c4474 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:32.974699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cf6ab-d88f-41aa-8784-2d91364c4474 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:32.975126 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cf6ab-d88f-41aa-8784-2d91364c4474 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:32.992346 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4cf6ab-d88f-41aa-8784-2d91364c4474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:32.995490 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4cf6ab-d88f-41aa-8784-2d91364c4474 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:33.014266 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 239/467] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:35:32 2022] POST /identity/v3/users => generated 246 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-2e4cf6ab-d88f-41aa-8784-2d91364c4474 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:33.014565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:33.015547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:33.016410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:33.016410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:33.016410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:33.018747 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 229/468] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:33.024918 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392f94c-e77f-4e93-a8a7-f4a3e77c8a20 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:33.059129 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f665ce3-a9ef-4dec-b274-671dee4e6960 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:33.059794 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f665ce3-a9ef-4dec-b274-671dee4e6960 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:33.060071 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f665ce3-a9ef-4dec-b274-671dee4e6960 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:33.061695 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 230/469] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:33.067651 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e2adb-3476-43fb-bc38-d498f59e285b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:33.067973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e2adb-3476-43fb-bc38-d498f59e285b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:33.068200 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e2adb-3476-43fb-bc38-d498f59e285b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:33.090997 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-442e2adb-3476-43fb-bc38-d498f59e285b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:33.092664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-442e2adb-3476-43fb-bc38-d498f59e285b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:33.206672 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 231/470] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:33.259921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392f94c-e77f-4e93-a8a7-f4a3e77c8a20 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:33.266494 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:33.308902 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876abec-6f38-4202-aee6-1435edb73f03 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:33.310357 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876abec-6f38-4202-aee6-1435edb73f03 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:33.310862 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876abec-6f38-4202-aee6-1435edb73f03 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:33.312507 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 232/471] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:35:33.318410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fac2c-5870-47cc-a047-3b5378c1ae3c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:33.319048 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fac2c-5870-47cc-a047-3b5378c1ae3c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:33.319519 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fac2c-5870-47cc-a047-3b5378c1ae3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:33.350020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-702fac2c-5870-47cc-a047-3b5378c1ae3c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:33.351917 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-702fac2c-5870-47cc-a047-3b5378c1ae3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:33.353345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2197b-cf1c-4754-b303-0cc1320e07b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:33.354805 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2197b-cf1c-4754-b303-0cc1320e07b5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:33.355347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2197b-cf1c-4754-b303-0cc1320e07b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:33.355638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2197b-cf1c-4754-b303-0cc1320e07b5 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:33.356280 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2197b-cf1c-4754-b303-0cc1320e07b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:33.356581 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2197b-cf1c-4754-b303-0cc1320e07b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:33.358020 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2197b-cf1c-4754-b303-0cc1320e07b5 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:33.376888 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:33.396839 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 240/472] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:33.406068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2197b-cf1c-4754-b303-0cc1320e07b5 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:33.460867 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 233/473] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:33.520672 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fac2c-5870-47cc-a047-3b5378c1ae3c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:33.571204 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2197b-cf1c-4754-b303-0cc1320e07b5 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:33.611577 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea7f48-be8d-42e4-969a-674d29c1dd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:33.612521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea7f48-be8d-42e4-969a-674d29c1dd95 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:33.612981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea7f48-be8d-42e4-969a-674d29c1dd95 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:33.613437 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea7f48-be8d-42e4-969a-674d29c1dd95 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:33.614403 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea7f48-be8d-42e4-969a-674d29c1dd95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:33.616196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea7f48-be8d-42e4-969a-674d29c1dd95 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:33.652485 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 241/474] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:35:33 2022] POST /identity/v3/users => generated 248 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:35:33.716229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fac2c-5870-47cc-a047-3b5378c1ae3c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:33.751334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea7f48-be8d-42e4-969a-674d29c1dd95 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:33.778996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4292c2-5f73-47a0-9464-5c03a2de4c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:33.779963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4292c2-5f73-47a0-9464-5c03a2de4c0c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:33.781182 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4292c2-5f73-47a0-9464-5c03a2de4c0c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:33.782274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4292c2-5f73-47a0-9464-5c03a2de4c0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:33.799611 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:33.813350 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4292c2-5f73-47a0-9464-5c03a2de4c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:33.818216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4292c2-5f73-47a0-9464-5c03a2de4c0c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:33.839367 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-dd4292c2-5f73-47a0-9464-5c03a2de4c0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:33.853149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:33.854740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:33.855574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:33.855574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:33.855574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:33.855574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 234/475] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:33.867244 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4292c2-5f73-47a0-9464-5c03a2de4c0c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:33.936114 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea7f48-be8d-42e4-969a-674d29c1dd95 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:33.975972 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68c49f-76a2-46bb-9b8f-22b642c2771b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:33.976942 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68c49f-76a2-46bb-9b8f-22b642c2771b None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:33.977417 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68c49f-76a2-46bb-9b8f-22b642c2771b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:33.977983 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68c49f-76a2-46bb-9b8f-22b642c2771b None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:33.978897 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68c49f-76a2-46bb-9b8f-22b642c2771b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:33.981198 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68c49f-76a2-46bb-9b8f-22b642c2771b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:33.990614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.sql.core [None req-1a68c49f-76a2-46bb-9b8f-22b642c2771b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 12:35:33.990614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 12:35:33.990614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [parameters: {'id': 'a7b4eadd30c0407783d5118fdc552fb1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 12:35:33.990614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93092) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-1a68c49f-76a2-46bb-9b8f-22b642c2771b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application conn.query(q) Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application result.read() Oct 04 12:35:34.010140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return ref.to_dict() Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 12:35:34.011630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application next(self.gen) Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application yield resource Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application next(self.gen) Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise self.value Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application session.commit() Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 12:35:34.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self.session.flush() Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application compat.raise_( Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 12:35:34.013658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 12:35:34.014650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application conn.query(q) Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application result.read() Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 12:35:34.015619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [parameters: {'id': 'a7b4eadd30c0407783d5118fdc552fb1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:34.016527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 12:35:34.017500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:34.020448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 242/476] 10.222.0.33 () {60 vars in 1131 bytes} [Tue Oct 4 12:35:33 2022] POST /identity/v3/roles => generated 144 bytes in 269 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:34.026662 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68c49f-76a2-46bb-9b8f-22b642c2771b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:34.050930 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4292c2-5f73-47a0-9464-5c03a2de4c0c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:34.090457 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124b0e4-1629-46bd-864d-3f787d93f884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:34.091004 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124b0e4-1629-46bd-864d-3f787d93f884 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:34.091283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124b0e4-1629-46bd-864d-3f787d93f884 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:34.091555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124b0e4-1629-46bd-864d-3f787d93f884 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:34.092625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124b0e4-1629-46bd-864d-3f787d93f884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:34.092625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124b0e4-1629-46bd-864d-3f787d93f884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:34.094140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124b0e4-1629-46bd-864d-3f787d93f884 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:34.109725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 235/477] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:34.121176 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124b0e4-1629-46bd-864d-3f787d93f884 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:34.147399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:34.174970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68c49f-76a2-46bb-9b8f-22b642c2771b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:34.218687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:34.229262 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b47d0-57d4-4bee-880d-a1208515b03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:34.230286 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b47d0-57d4-4bee-880d-a1208515b03d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:34.230634 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b47d0-57d4-4bee-880d-a1208515b03d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:34.230971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b47d0-57d4-4bee-880d-a1208515b03d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:34.245310 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b47d0-57d4-4bee-880d-a1208515b03d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-9b2b47d0-57d4-4bee-880d-a1208515b03d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:34.252649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:34.253638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:34.254472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 243/478] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:35:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:34.266093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b47d0-57d4-4bee-880d-a1208515b03d None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:34.298380 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124b0e4-1629-46bd-864d-3f787d93f884 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:34.342107 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365ca0c-9a50-4a35-9ea6-db1ae1870d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:34.342715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365ca0c-9a50-4a35-9ea6-db1ae1870d17 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:34.342942 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365ca0c-9a50-4a35-9ea6-db1ae1870d17 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:34.343189 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365ca0c-9a50-4a35-9ea6-db1ae1870d17 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:34.343858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365ca0c-9a50-4a35-9ea6-db1ae1870d17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:34.345346 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365ca0c-9a50-4a35-9ea6-db1ae1870d17 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:34.374839 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 236/479] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:35:34 2022] POST /identity/v3/users => generated 249 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:35:34.506762 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b47d0-57d4-4bee-880d-a1208515b03d None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:34.568570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b65a04-27c1-4130-925a-677acbafe5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:34.569197 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b65a04-27c1-4130-925a-677acbafe5d2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:34.569473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b65a04-27c1-4130-925a-677acbafe5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:34.569738 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b65a04-27c1-4130-925a-677acbafe5d2 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:34.570464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b65a04-27c1-4130-925a-677acbafe5d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:34.570765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b65a04-27c1-4130-925a-677acbafe5d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:34.572217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b65a04-27c1-4130-925a-677acbafe5d2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:34.599446 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 244/480] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:35.648882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db06ae-8c67-4b84-b440-0a6bed0a8260 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:35.649377 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db06ae-8c67-4b84-b440-0a6bed0a8260 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:35.649377 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db06ae-8c67-4b84-b440-0a6bed0a8260 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:35.651177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 237/481] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:35.656592 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fc021-131d-4a26-82e0-a4a77770e35c None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:35.657110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fc021-131d-4a26-82e0-a4a77770e35c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:35.657110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fc021-131d-4a26-82e0-a4a77770e35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:35.664931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:35.679619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-708fc021-131d-4a26-82e0-a4a77770e35c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:35.680860 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-708fc021-131d-4a26-82e0-a4a77770e35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:35.763745 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 245/482] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:36.200891 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db06ae-8c67-4b84-b440-0a6bed0a8260 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:36.377364 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db06ae-8c67-4b84-b440-0a6bed0a8260 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:36.402203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:36.444710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd98fb-ef43-445b-bbcf-baa0cc1bed56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:36.445836 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd98fb-ef43-445b-bbcf-baa0cc1bed56 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:36.446345 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd98fb-ef43-445b-bbcf-baa0cc1bed56 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:36.446817 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd98fb-ef43-445b-bbcf-baa0cc1bed56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:36.467915 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd98fb-ef43-445b-bbcf-baa0cc1bed56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-6fbd98fb-ef43-445b-bbcf-baa0cc1bed56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:36.477849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:36.478874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:36.480216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:36.480216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:36.480216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:36.480216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:36.480216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:36.480216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:36.480216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:36.480216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:36.480216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:36.480216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:36.480216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:36.480216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:36.481302 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 238/483] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:36.490612 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fc021-131d-4a26-82e0-a4a77770e35c None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:36.570911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745d835-85a8-4268-99a3-693e5b324998 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:36.571392 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745d835-85a8-4268-99a3-693e5b324998 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:36.571883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745d835-85a8-4268-99a3-693e5b324998 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:36.575092 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 239/484] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:36.583622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6206a-c373-4559-9a10-1912d2e95ebe None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:36.584111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6206a-c373-4559-9a10-1912d2e95ebe None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:36.584588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6206a-c373-4559-9a10-1912d2e95ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:36.621305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a9e6206a-c373-4559-9a10-1912d2e95ebe None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:36.624260 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a9e6206a-c373-4559-9a10-1912d2e95ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:36.749367 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fc021-131d-4a26-82e0-a4a77770e35c None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:36.765841 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 240/485] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:36.815091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5aad9-d47e-4fc4-8e03-7fa385ae667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:36.815704 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5aad9-d47e-4fc4-8e03-7fa385ae667f None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:36.815981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5aad9-d47e-4fc4-8e03-7fa385ae667f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:36.816250 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df5aad9-d47e-4fc4-8e03-7fa385ae667f None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:36.816922 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5aad9-d47e-4fc4-8e03-7fa385ae667f None admin] RBAC: Adding query filter params (name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:36.817223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5aad9-d47e-4fc4-8e03-7fa385ae667f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:36.818655 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df5aad9-d47e-4fc4-8e03-7fa385ae667f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:36.820191 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:36.834125 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 246/486] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:36.843865 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6206a-c373-4559-9a10-1912d2e95ebe None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:36.984742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6206a-c373-4559-9a10-1912d2e95ebe None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:36.993361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:37.028113 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df933bdb-4b07-4ff0-9436-110a49d6cd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:37.029156 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df933bdb-4b07-4ff0-9436-110a49d6cd5a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:37.029634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df933bdb-4b07-4ff0-9436-110a49d6cd5a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:37.030240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df933bdb-4b07-4ff0-9436-110a49d6cd5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:37.041958 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df933bdb-4b07-4ff0-9436-110a49d6cd5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:37.043800 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df933bdb-4b07-4ff0-9436-110a49d6cd5a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-df933bdb-4b07-4ff0-9436-110a49d6cd5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:37.060181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:37.061133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:37.061966 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:37.061966 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:37.061966 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:37.064246 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 241/487] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:37.072613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5aad9-d47e-4fc4-8e03-7fa385ae667f None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:37.244781 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df5aad9-d47e-4fc4-8e03-7fa385ae667f None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:37.311652 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f4045-d41d-449a-81ea-908a7316f09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:37.312199 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f4045-d41d-449a-81ea-908a7316f09b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:37.312478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f4045-d41d-449a-81ea-908a7316f09b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:37.312751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f4045-d41d-449a-81ea-908a7316f09b None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:37.313409 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f4045-d41d-449a-81ea-908a7316f09b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:37.313711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f4045-d41d-449a-81ea-908a7316f09b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:37.315154 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f4045-d41d-449a-81ea-908a7316f09b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:37.336541 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 247/488] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:37.345077 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df933bdb-4b07-4ff0-9436-110a49d6cd5a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:37.352173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f4045-d41d-449a-81ea-908a7316f09b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:37.459417 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:37.564643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f4045-d41d-449a-81ea-908a7316f09b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:37.574584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df933bdb-4b07-4ff0-9436-110a49d6cd5a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:37.610153 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6052e-a036-4d03-86f5-e0c55edd7d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:37.611200 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6052e-a036-4d03-86f5-e0c55edd7d1a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:37.611702 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6052e-a036-4d03-86f5-e0c55edd7d1a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:37.612203 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6052e-a036-4d03-86f5-e0c55edd7d1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:37.622241 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1ae0e-e101-4915-81df-0b91208e0be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:37.622888 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1ae0e-e101-4915-81df-0b91208e0be9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:37.623153 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1ae0e-e101-4915-81df-0b91208e0be9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:37.623419 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1ae0e-e101-4915-81df-0b91208e0be9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:37.631910 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1ae0e-e101-4915-81df-0b91208e0be9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:37.632319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6052e-a036-4d03-86f5-e0c55edd7d1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:37.633371 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1ae0e-e101-4915-81df-0b91208e0be9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-45a6052e-a036-4d03-86f5-e0c55edd7d1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:37.642933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:37.644010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 248/489] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:37.652879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42800-a4ac-4793-b888-fe23a60b56e2 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:37.654640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42800-a4ac-4793-b888-fe23a60b56e2 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:37.654640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42800-a4ac-4793-b888-fe23a60b56e2 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-10d1ae0e-e101-4915-81df-0b91208e0be9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:37.654758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:37.657812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 242/490] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9f066-3aac-4318-938f-f6a0f86e323a None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:37.664117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9f066-3aac-4318-938f-f6a0f86e323a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:37.666937 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9f066-3aac-4318-938f-f6a0f86e323a None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:37.666937 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 243/491] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:37.667112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 249/492] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:37.667266 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9f066-3aac-4318-938f-f6a0f86e323a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:37.667406 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bbaa0-9fb5-4b70-bacd-eba342a0ae3f None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:37.667406 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bbaa0-9fb5-4b70-bacd-eba342a0ae3f None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:37.667406 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008bbaa0-9fb5-4b70-bacd-eba342a0ae3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:37.680456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:37.689896 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-008bbaa0-9fb5-4b70-bacd-eba342a0ae3f None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:37.691506 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-008bbaa0-9fb5-4b70-bacd-eba342a0ae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:37.777683 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 250/493] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:37.778987 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6c380-152b-4080-996f-8e5ec229616e None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:37.779272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6c380-152b-4080-996f-8e5ec229616e None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:37.779546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6c380-152b-4080-996f-8e5ec229616e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:37.782682 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9f066-3aac-4318-938f-f6a0f86e323a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:37.801578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-15e6c380-152b-4080-996f-8e5ec229616e None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:37.802858 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-15e6c380-152b-4080-996f-8e5ec229616e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:37.854780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292151-b117-4e2e-930e-1074d0403541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:37.855970 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292151-b117-4e2e-930e-1074d0403541 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:37.856345 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292151-b117-4e2e-930e-1074d0403541 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:37.856813 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292151-b117-4e2e-930e-1074d0403541 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:37.858205 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:37.859861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6292151-b117-4e2e-930e-1074d0403541 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:37.860184 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6292151-b117-4e2e-930e-1074d0403541 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:37.862743 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6292151-b117-4e2e-930e-1074d0403541 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:37.898067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 251/494] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:37.898067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6c380-152b-4080-996f-8e5ec229616e None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:37.934270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 244/495] 10.222.0.33 () {56 vars in 1137 bytes} [Tue Oct 4 12:35:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:37.936026 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1066681-fd03-4cfe-9484-c8e60d857264 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:37.936318 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1066681-fd03-4cfe-9484-c8e60d857264 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:37.936669 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1066681-fd03-4cfe-9484-c8e60d857264 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:37.938216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 245/496] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:37.943064 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1066681-fd03-4cfe-9484-c8e60d857264 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:38.030290 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6c380-152b-4080-996f-8e5ec229616e None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:38.061725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:38.070398 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32020bd0-4ad7-4b74-b687-9244338e242b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:38.071010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32020bd0-4ad7-4b74-b687-9244338e242b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:38.071284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32020bd0-4ad7-4b74-b687-9244338e242b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:38.071561 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32020bd0-4ad7-4b74-b687-9244338e242b None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:38.072211 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32020bd0-4ad7-4b74-b687-9244338e242b None admin] RBAC: Adding query filter params (name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:38.072508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32020bd0-4ad7-4b74-b687-9244338e242b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:38.074659 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32020bd0-4ad7-4b74-b687-9244338e242b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:38.083584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 252/497] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:38.089891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fe743-e505-4434-875a-a11f38c835fd None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:38.090227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fe743-e505-4434-875a-a11f38c835fd None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:38.090534 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fe743-e505-4434-875a-a11f38c835fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:38.093565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1066681-fd03-4cfe-9484-c8e60d857264 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:38.108609 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-863fe743-e505-4434-875a-a11f38c835fd None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:38.109874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-863fe743-e505-4434-875a-a11f38c835fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:38.136048 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6d779-881e-47ec-b39e-7c2b47d58115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:38.136670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6d779-881e-47ec-b39e-7c2b47d58115 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:38.136966 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6d779-881e-47ec-b39e-7c2b47d58115 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:38.137211 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6d779-881e-47ec-b39e-7c2b47d58115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:38.149187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:38.214070 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 253/498] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:38.214070 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fe743-e505-4434-875a-a11f38c835fd None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:38.245982 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6d779-881e-47ec-b39e-7c2b47d58115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:38.248065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6d779-881e-47ec-b39e-7c2b47d58115 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:38.363060 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:38.381488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fe743-e505-4434-875a-a11f38c835fd None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:38.404478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 246/499] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:35:38.405305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6d779-881e-47ec-b39e-7c2b47d58115 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:38.438716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19182b9e-74d5-4de7-9cad-71346c355c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:38.439570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19182b9e-74d5-4de7-9cad-71346c355c60 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:38.439931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19182b9e-74d5-4de7-9cad-71346c355c60 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:38.440208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19182b9e-74d5-4de7-9cad-71346c355c60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:38.459159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19182b9e-74d5-4de7-9cad-71346c355c60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-19182b9e-74d5-4de7-9cad-71346c355c60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:38.468940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:38.470208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:38.471144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 254/500] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:38.472194 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19182b9e-74d5-4de7-9cad-71346c355c60 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:38.508968 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:38.542885 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6d779-881e-47ec-b39e-7c2b47d58115 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:38.581914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bffde-c3b7-47a3-ad98-ce1c4021f124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:38.582584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bffde-c3b7-47a3-ad98-ce1c4021f124 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:38.582853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bffde-c3b7-47a3-ad98-ce1c4021f124 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:38.583129 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bffde-c3b7-47a3-ad98-ce1c4021f124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:38.594293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bffde-c3b7-47a3-ad98-ce1c4021f124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:38.595824 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bffde-c3b7-47a3-ad98-ce1c4021f124 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:38.608282 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19182b9e-74d5-4de7-9cad-71346c355c60 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-d82bffde-c3b7-47a3-ad98-ce1c4021f124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:38.611664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:38.612575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:38.613450 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:38.613450 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:38.613450 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:38.613450 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 247/501] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:38.613672 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bffde-c3b7-47a3-ad98-ce1c4021f124 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:38.648509 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:38.667182 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9fba5-9562-494e-bdf9-4d3aad300dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:38.668055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9fba5-9562-494e-bdf9-4d3aad300dad None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:38.668442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9fba5-9562-494e-bdf9-4d3aad300dad None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:38.668789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9fba5-9562-494e-bdf9-4d3aad300dad None admin] PATH_INFO: `/v3/roles/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:38.684052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9fba5-9562-494e-bdf9-4d3aad300dad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-36d9fba5-9562-494e-bdf9-4d3aad300dad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:38.689055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:38.690013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:38.690885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 255/502] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:38.692150 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9fba5-9562-494e-bdf9-4d3aad300dad None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:38.750791 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bffde-c3b7-47a3-ad98-ce1c4021f124 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:38.787693 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:38.789721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f8a13-fb35-4e3b-93e8-fb186955d960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:38.790390 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f8a13-fb35-4e3b-93e8-fb186955d960 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:38.790664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f8a13-fb35-4e3b-93e8-fb186955d960 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:38.790944 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f8a13-fb35-4e3b-93e8-fb186955d960 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:38.803120 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f8a13-fb35-4e3b-93e8-fb186955d960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:38.804621 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f8a13-fb35-4e3b-93e8-fb186955d960 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:38.806542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9fba5-9562-494e-bdf9-4d3aad300dad None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-6d3f8a13-fb35-4e3b-93e8-fb186955d960 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:38.822065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:38.823142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:38.824012 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:38.824012 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:38.824012 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:38.824012 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 248/503] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:38.824436 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f8a13-fb35-4e3b-93e8-fb186955d960 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:38.847043 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905c4ab-0823-42b2-808e-2f4b9df14b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:38.847646 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905c4ab-0823-42b2-808e-2f4b9df14b25 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:38.847923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905c4ab-0823-42b2-808e-2f4b9df14b25 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:38.848225 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905c4ab-0823-42b2-808e-2f4b9df14b25 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:38.848902 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905c4ab-0823-42b2-808e-2f4b9df14b25 None admin] RBAC: Adding query filter params (name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:38.849205 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905c4ab-0823-42b2-808e-2f4b9df14b25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:38.850667 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905c4ab-0823-42b2-808e-2f4b9df14b25 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:38.862180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 256/504] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:38.863261 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905c4ab-0823-42b2-808e-2f4b9df14b25 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:38.918915 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:38.948165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f8a13-fb35-4e3b-93e8-fb186955d960 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:38.981924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905c4ab-0823-42b2-808e-2f4b9df14b25 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:38.991608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca767f8d-0624-4834-b068-7427a47c7759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:38.992236 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca767f8d-0624-4834-b068-7427a47c7759 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:38.992508 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca767f8d-0624-4834-b068-7427a47c7759 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:38.992906 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca767f8d-0624-4834-b068-7427a47c7759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:39.004849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca767f8d-0624-4834-b068-7427a47c7759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:39.006386 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca767f8d-0624-4834-b068-7427a47c7759 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-ca767f8d-0624-4834-b068-7427a47c7759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:39.023063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:39.025104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:39.027310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:39.027310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:39.027310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:39.027640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4234bb-dd53-443c-a024-53e71b98c0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:39.027640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4234bb-dd53-443c-a024-53e71b98c0a5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:39.027640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4234bb-dd53-443c-a024-53e71b98c0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:39.027640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4234bb-dd53-443c-a024-53e71b98c0a5 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:39.028240 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4234bb-dd53-443c-a024-53e71b98c0a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:39.028240 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4234bb-dd53-443c-a024-53e71b98c0a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:39.028240 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4234bb-dd53-443c-a024-53e71b98c0a5 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:39.028578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 249/505] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:39.029892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca767f8d-0624-4834-b068-7427a47c7759 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:39.042264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 257/506] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:39.043119 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4234bb-dd53-443c-a024-53e71b98c0a5 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:39.057576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:39.144188 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca767f8d-0624-4834-b068-7427a47c7759 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:39.166392 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4234bb-dd53-443c-a024-53e71b98c0a5 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:39.175054 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:39.181627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6305f-d439-4d97-a72b-29a011b40df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:39.182257 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6305f-d439-4d97-a72b-29a011b40df5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:39.182533 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6305f-d439-4d97-a72b-29a011b40df5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:39.182802 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6305f-d439-4d97-a72b-29a011b40df5 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:39.183468 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6305f-d439-4d97-a72b-29a011b40df5 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:39.183767 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6305f-d439-4d97-a72b-29a011b40df5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:39.185165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6305f-d439-4d97-a72b-29a011b40df5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:39.193482 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 250/507] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:39.194351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6305f-d439-4d97-a72b-29a011b40df5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:39.209002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b37aa2-50c4-4605-92e7-0d6835afb639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:39.209611 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b37aa2-50c4-4605-92e7-0d6835afb639 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:39.209897 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b37aa2-50c4-4605-92e7-0d6835afb639 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:39.210201 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b37aa2-50c4-4605-92e7-0d6835afb639 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:39.210853 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b37aa2-50c4-4605-92e7-0d6835afb639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:39.211186 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b37aa2-50c4-4605-92e7-0d6835afb639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:39.212584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b37aa2-50c4-4605-92e7-0d6835afb639 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:39.224912 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 258/508] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:39.224912 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b37aa2-50c4-4605-92e7-0d6835afb639 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:39.305438 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:39.330781 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6305f-d439-4d97-a72b-29a011b40df5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:39.341357 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b37aa2-50c4-4605-92e7-0d6835afb639 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:39.367940 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c597ce0-15aa-4c00-910e-2d14dd1f8b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:39.368717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c597ce0-15aa-4c00-910e-2d14dd1f8b22 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:39.368994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c597ce0-15aa-4c00-910e-2d14dd1f8b22 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:39.369263 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c597ce0-15aa-4c00-910e-2d14dd1f8b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:39.381459 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c597ce0-15aa-4c00-910e-2d14dd1f8b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:39.382123 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600a496-7fcf-48dc-97fe-9769ea3c1491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:39.382796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600a496-7fcf-48dc-97fe-9769ea3c1491 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:39.382908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c597ce0-15aa-4c00-910e-2d14dd1f8b22 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:39.383067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600a496-7fcf-48dc-97fe-9769ea3c1491 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:39.383525 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600a496-7fcf-48dc-97fe-9769ea3c1491 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:39.383976 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600a496-7fcf-48dc-97fe-9769ea3c1491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:39.384276 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600a496-7fcf-48dc-97fe-9769ea3c1491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:39.386241 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600a496-7fcf-48dc-97fe-9769ea3c1491 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-6c597ce0-15aa-4c00-910e-2d14dd1f8b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:39.399738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:39.401516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:39.402405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:39.402405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:39.402405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:39.402405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 251/509] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:39.402405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c597ce0-15aa-4c00-910e-2d14dd1f8b22 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:39.403189 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 259/510] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:39.404030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600a496-7fcf-48dc-97fe-9769ea3c1491 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:39.464761 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:39.530205 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600a496-7fcf-48dc-97fe-9769ea3c1491 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:39.535395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c597ce0-15aa-4c00-910e-2d14dd1f8b22 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:39.564131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:39.570455 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2b1a9-314e-4391-8315-51c6d1f4c846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:39.571081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2b1a9-314e-4391-8315-51c6d1f4c846 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:39.571352 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2b1a9-314e-4391-8315-51c6d1f4c846 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:39.571613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2b1a9-314e-4391-8315-51c6d1f4c846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:39.574827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c635102-6a5e-4d08-b487-c4fbaad19f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:39.575682 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c635102-6a5e-4d08-b487-c4fbaad19f4f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:39.576021 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c635102-6a5e-4d08-b487-c4fbaad19f4f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:39.576358 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c635102-6a5e-4d08-b487-c4fbaad19f4f None admin] PATH_INFO: `/v3/users/nova` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:39.583310 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2b1a9-314e-4391-8315-51c6d1f4c846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:39.584590 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c635102-6a5e-4d08-b487-c4fbaad19f4f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:39.584803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2b1a9-314e-4391-8315-51c6d1f4c846 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:39.586057 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c635102-6a5e-4d08-b487-c4fbaad19f4f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-8c635102-6a5e-4d08-b487-c4fbaad19f4f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:39.599608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:39.607410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:39.608335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:39.608335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:39.608335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:39.608335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:39.608335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:39.608335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:39.608335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:39.608335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:39.608335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:35:39.608335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-48c2b1a9-314e-4391-8315-51c6d1f4c846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:39.608847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:39.609617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:39.610508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:39.610508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:39.610508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:39.610508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 260/511] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:39.610715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 252/512] 10.222.0.33 () {56 vars in 1093 bytes} [Tue Oct 4 12:35:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:39.610781 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2b1a9-314e-4391-8315-51c6d1f4c846 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:39.610855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c635102-6a5e-4d08-b487-c4fbaad19f4f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:39.707808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2b1a9-314e-4391-8315-51c6d1f4c846 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:39.719858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c635102-6a5e-4d08-b487-c4fbaad19f4f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:39.724143 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:39.743329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469f5a4-7f12-42ac-a062-33330dd4abce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:39.743955 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469f5a4-7f12-42ac-a062-33330dd4abce None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:39.744227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469f5a4-7f12-42ac-a062-33330dd4abce None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:39.745014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469f5a4-7f12-42ac-a062-33330dd4abce None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:39.752103 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469f5a4-7f12-42ac-a062-33330dd4abce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:39.753560 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469f5a4-7f12-42ac-a062-33330dd4abce None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-d469f5a4-7f12-42ac-a062-33330dd4abce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:35:39.767326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:39.768253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:39.769080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:35:39.769080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:35:39.769080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:35:39.769080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:35:39.769080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:35:39.769080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:35:39.769080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:39.769080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:39.769080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 261/513] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:39.769816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469f5a4-7f12-42ac-a062-33330dd4abce None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:39.775621 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc70b8-d3d9-4ecc-ac6b-173051fbb10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:39.776285 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc70b8-d3d9-4ecc-ac6b-173051fbb10a None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:39.776563 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc70b8-d3d9-4ecc-ac6b-173051fbb10a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:39.776834 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc70b8-d3d9-4ecc-ac6b-173051fbb10a None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:39.777497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc70b8-d3d9-4ecc-ac6b-173051fbb10a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:39.778983 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc70b8-d3d9-4ecc-ac6b-173051fbb10a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:39.801173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 253/514] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:35:39 2022] POST /identity/v3/users => generated 247 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:35:39.802019 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc70b8-d3d9-4ecc-ac6b-173051fbb10a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:39.834951 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:39.915025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469f5a4-7f12-42ac-a062-33330dd4abce None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:39.956044 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc70b8-d3d9-4ecc-ac6b-173051fbb10a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:39.974804 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a84108-7a8d-4b61-adcd-5f3408657f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:39.975347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a84108-7a8d-4b61-adcd-5f3408657f06 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:39.975625 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a84108-7a8d-4b61-adcd-5f3408657f06 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:39.975892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a84108-7a8d-4b61-adcd-5f3408657f06 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:39.976541 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a84108-7a8d-4b61-adcd-5f3408657f06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:39.976841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a84108-7a8d-4b61-adcd-5f3408657f06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:39.978695 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a84108-7a8d-4b61-adcd-5f3408657f06 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:40.005363 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 262/515] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:40.006364 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a84108-7a8d-4b61-adcd-5f3408657f06 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:40.052358 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab2cbe-8526-4116-a7fc-1b779e2a71a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:40.053546 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab2cbe-8526-4116-a7fc-1b779e2a71a0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:40.053546 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab2cbe-8526-4116-a7fc-1b779e2a71a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:40.056470 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab2cbe-8526-4116-a7fc-1b779e2a71a0 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:40.056470 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab2cbe-8526-4116-a7fc-1b779e2a71a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:40.056470 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab2cbe-8526-4116-a7fc-1b779e2a71a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:40.058536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab2cbe-8526-4116-a7fc-1b779e2a71a0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:40.097831 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 254/516] 10.222.0.33 () {56 vars in 1133 bytes} [Tue Oct 4 12:35:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:40.097831 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab2cbe-8526-4116-a7fc-1b779e2a71a0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:40.134583 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:40.158307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a84108-7a8d-4b61-adcd-5f3408657f06 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:40.160222 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:40.210154 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d34ff0-5093-49ab-a9c0-8fed9d38bd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:40.210774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34ff0-5093-49ab-a9c0-8fed9d38bd78 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:40.211053 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34ff0-5093-49ab-a9c0-8fed9d38bd78 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:40.211322 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34ff0-5093-49ab-a9c0-8fed9d38bd78 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:40.212028 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d34ff0-5093-49ab-a9c0-8fed9d38bd78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:40.212331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d34ff0-5093-49ab-a9c0-8fed9d38bd78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:40.213968 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d34ff0-5093-49ab-a9c0-8fed9d38bd78 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:40.230718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 263/517] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:40.230718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d34ff0-5093-49ab-a9c0-8fed9d38bd78 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:40.309209 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab2cbe-8526-4116-a7fc-1b779e2a71a0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:40.379577 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a65ba5-8f48-4b27-abff-2892539f9165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:40.380204 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a65ba5-8f48-4b27-abff-2892539f9165 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:40.380482 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a65ba5-8f48-4b27-abff-2892539f9165 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:40.380762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a65ba5-8f48-4b27-abff-2892539f9165 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:40.381648 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a65ba5-8f48-4b27-abff-2892539f9165 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:40.381750 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d34ff0-5093-49ab-a9c0-8fed9d38bd78 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:40.381828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a65ba5-8f48-4b27-abff-2892539f9165 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:40.383927 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a65ba5-8f48-4b27-abff-2892539f9165 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:40.406204 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 255/518] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:35:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:40.407076 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a65ba5-8f48-4b27-abff-2892539f9165 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:40.429790 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5d717-215f-4f42-958a-c993bf8e1009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:40.430467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5d717-215f-4f42-958a-c993bf8e1009 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:40.430743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5d717-215f-4f42-958a-c993bf8e1009 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:40.431017 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5d717-215f-4f42-958a-c993bf8e1009 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:40.439635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5d717-215f-4f42-958a-c993bf8e1009 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:40.441131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5d717-215f-4f42-958a-c993bf8e1009 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-a4b5d717-215f-4f42-958a-c993bf8e1009 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:40.454595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:40.455576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:40.456454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:40.456454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:40.456454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:40.456454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:40.456454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:40.456454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:40.456454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:40.456454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:40.456454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 12:35:40.456454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:40.456454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 264/519] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:40.456996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5d717-215f-4f42-958a-c993bf8e1009 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:40.489139 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:40.514903 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:40.543007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a65ba5-8f48-4b27-abff-2892539f9165 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:40.578543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5d717-215f-4f42-958a-c993bf8e1009 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:40.590113 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4383553-1178-477e-80cd-ba7c89d48b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:40.591278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4383553-1178-477e-80cd-ba7c89d48b5f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:40.591878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4383553-1178-477e-80cd-ba7c89d48b5f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:40.592638 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4383553-1178-477e-80cd-ba7c89d48b5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:40.618206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702d3b0-7aac-4191-a0d3-05676462f22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:40.618844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702d3b0-7aac-4191-a0d3-05676462f22e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:40.619113 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702d3b0-7aac-4191-a0d3-05676462f22e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:40.619376 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702d3b0-7aac-4191-a0d3-05676462f22e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:40.628469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702d3b0-7aac-4191-a0d3-05676462f22e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:40.629943 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702d3b0-7aac-4191-a0d3-05676462f22e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-3702d3b0-7aac-4191-a0d3-05676462f22e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:40.641690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:40.643624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:40.645785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:40.645785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:40.645785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:40.645785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:40.645785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:40.645785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:40.645785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:40.645785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:40.645785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 12:35:40.645785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:40.647892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 265/520] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:40.648706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702d3b0-7aac-4191-a0d3-05676462f22e None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:40.694557 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4383553-1178-477e-80cd-ba7c89d48b5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:40.696278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4383553-1178-477e-80cd-ba7c89d48b5f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:40.739965 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:40.768559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702d3b0-7aac-4191-a0d3-05676462f22e None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:40.790937 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:40.813134 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc08ca2-63ad-47d4-af00-f62dc7fabf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:40.813588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc08ca2-63ad-47d4-af00-f62dc7fabf31 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:40.813872 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc08ca2-63ad-47d4-af00-f62dc7fabf31 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:40.814196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc08ca2-63ad-47d4-af00-f62dc7fabf31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:40.814959 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc08ca2-63ad-47d4-af00-f62dc7fabf31 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=f2f4f9541f89433cad52e826794b049f) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:40.815272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc08ca2-63ad-47d4-af00-f62dc7fabf31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:40.816817 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc08ca2-63ad-47d4-af00-f62dc7fabf31 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:40.819022 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 256/521] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:35:40.819951 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4383553-1178-477e-80cd-ba7c89d48b5f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:40.829607 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 266/522] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:35:40 2022] GET /identity/v3/role_assignments?user.id=f2f4f9541f89433cad52e826794b049f&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:40.830315 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc08ca2-63ad-47d4-af00-f62dc7fabf31 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:40.978342 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4383553-1178-477e-80cd-ba7c89d48b5f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:40.986267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc08ca2-63ad-47d4-af00-f62dc7fabf31 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:41.020650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bd3b7-83f9-4b4d-81b6-7bf331ae2309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:41.021277 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bd3b7-83f9-4b4d-81b6-7bf331ae2309 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:41.022096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bd3b7-83f9-4b4d-81b6-7bf331ae2309 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:41.022387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bd3b7-83f9-4b4d-81b6-7bf331ae2309 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:41.023213 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bd3b7-83f9-4b4d-81b6-7bf331ae2309 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:41.023522 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bd3b7-83f9-4b4d-81b6-7bf331ae2309 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:41.025059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bd3b7-83f9-4b4d-81b6-7bf331ae2309 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:41.028229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a48bd-ccc3-49b0-9212-5ef71d2e794c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:41.028924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a48bd-ccc3-49b0-9212-5ef71d2e794c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:41.029274 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a48bd-ccc3-49b0-9212-5ef71d2e794c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:41.029611 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a48bd-ccc3-49b0-9212-5ef71d2e794c None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:41.030484 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a48bd-ccc3-49b0-9212-5ef71d2e794c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:41.031625 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a48bd-ccc3-49b0-9212-5ef71d2e794c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:41.033185 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a48bd-ccc3-49b0-9212-5ef71d2e794c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:41.047332 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 257/523] 10.222.0.33 () {56 vars in 1139 bytes} [Tue Oct 4 12:35:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:41.049235 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bd3b7-83f9-4b4d-81b6-7bf331ae2309 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:41.079037 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 267/524] 10.222.0.33 () {56 vars in 1137 bytes} [Tue Oct 4 12:35:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:41.080186 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a48bd-ccc3-49b0-9212-5ef71d2e794c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:41.087920 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:41.121518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:41.217903 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a48bd-ccc3-49b0-9212-5ef71d2e794c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:41.250260 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bd3b7-83f9-4b4d-81b6-7bf331ae2309 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:41.276368 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514af13-e5ec-46ab-b9e7-e75fb0d495c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:41.276878 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514af13-e5ec-46ab-b9e7-e75fb0d495c7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:41.277156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514af13-e5ec-46ab-b9e7-e75fb0d495c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:41.277421 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514af13-e5ec-46ab-b9e7-e75fb0d495c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:41.329710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbb467-af63-4506-b64c-7585190abbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:41.330873 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbb467-af63-4506-b64c-7585190abbdf None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:41.331348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbb467-af63-4506-b64c-7585190abbdf None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:41.331880 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbb467-af63-4506-b64c-7585190abbdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:41.350540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbb467-af63-4506-b64c-7585190abbdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:41.354881 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbb467-af63-4506-b64c-7585190abbdf None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:41.372397 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514af13-e5ec-46ab-b9e7-e75fb0d495c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:41.373950 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514af13-e5ec-46ab-b9e7-e75fb0d495c7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:41.376936 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-9acbb467-af63-4506-b64c-7585190abbdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:41.389995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:41.391920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:41.393809 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:41.393809 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:41.393809 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:41.400943 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 258/525] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:41.400943 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbb467-af63-4506-b64c-7585190abbdf None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:41.532022 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:41.560341 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbb467-af63-4506-b64c-7585190abbdf None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:41.569690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 268/526] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:35:41.571022 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514af13-e5ec-46ab-b9e7-e75fb0d495c7 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:41.604649 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d477311-722a-4c93-974b-a9c2b380826f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:41.605782 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d477311-722a-4c93-974b-a9c2b380826f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:41.605782 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d477311-722a-4c93-974b-a9c2b380826f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:41.606264 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d477311-722a-4c93-974b-a9c2b380826f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:41.693334 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d477311-722a-4c93-974b-a9c2b380826f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:41.694861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d477311-722a-4c93-974b-a9c2b380826f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:41.742899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514af13-e5ec-46ab-b9e7-e75fb0d495c7 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:41.748207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:41.805727 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:41.816617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b8717-25f1-4e30-980e-1b12e8f59ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:41.817516 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b8717-25f1-4e30-980e-1b12e8f59ac5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:41.818143 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b8717-25f1-4e30-980e-1b12e8f59ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:41.818705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b8717-25f1-4e30-980e-1b12e8f59ac5 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:41.820036 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b8717-25f1-4e30-980e-1b12e8f59ac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:41.820605 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b8717-25f1-4e30-980e-1b12e8f59ac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:41.822684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b8717-25f1-4e30-980e-1b12e8f59ac5 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:41.823569 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 259/527] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:35:41.824533 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d477311-722a-4c93-974b-a9c2b380826f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:41.849904 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 269/528] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:41.851647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b8717-25f1-4e30-980e-1b12e8f59ac5 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:42.006148 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d477311-722a-4c93-974b-a9c2b380826f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:42.014282 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b8717-25f1-4e30-980e-1b12e8f59ac5 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:42.063294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95513c-5eb6-41c7-9ee0-fe58c6d516ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:42.064617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95513c-5eb6-41c7-9ee0-fe58c6d516ed None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:42.065181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95513c-5eb6-41c7-9ee0-fe58c6d516ed None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:42.065864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95513c-5eb6-41c7-9ee0-fe58c6d516ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:42.077133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fe0b0-1df7-4fc8-803c-2f7d09834724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:42.077806 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fe0b0-1df7-4fc8-803c-2f7d09834724 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:42.078117 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fe0b0-1df7-4fc8-803c-2f7d09834724 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:42.078397 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fe0b0-1df7-4fc8-803c-2f7d09834724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:42.089055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95513c-5eb6-41c7-9ee0-fe58c6d516ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:42.092686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95513c-5eb6-41c7-9ee0-fe58c6d516ed None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:42.096851 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fe0b0-1df7-4fc8-803c-2f7d09834724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:42.099937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fe0b0-1df7-4fc8-803c-2f7d09834724 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-145fe0b0-1df7-4fc8-803c-2f7d09834724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:42.119329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:42.120923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:42.122643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:42.122643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:42.122643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:42.125494 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 270/529] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:42.126535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fe0b0-1df7-4fc8-803c-2f7d09834724 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-fa95513c-5eb6-41c7-9ee0-fe58c6d516ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:42.128995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:42.130909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:42.133286 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:42.133286 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:42.133286 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:42.133286 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 260/530] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:42.133286 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95513c-5eb6-41c7-9ee0-fe58c6d516ed None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:42.156903 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:42.213140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:42.275239 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fe0b0-1df7-4fc8-803c-2f7d09834724 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:42.310486 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95513c-5eb6-41c7-9ee0-fe58c6d516ed None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:42.318632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fa4bf-31e8-45f3-b0f8-72414fda0ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:42.319177 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fa4bf-31e8-45f3-b0f8-72414fda0ac0 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:42.319457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fa4bf-31e8-45f3-b0f8-72414fda0ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:42.319720 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fa4bf-31e8-45f3-b0f8-72414fda0ac0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:42.328069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fa4bf-31e8-45f3-b0f8-72414fda0ac0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:42.329566 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fa4bf-31e8-45f3-b0f8-72414fda0ac0 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-c42fa4bf-31e8-45f3-b0f8-72414fda0ac0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:35:42.342306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:42.344258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:42.346765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:35:42.346765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:35:42.346765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:35:42.346765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:35:42.346765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:35:42.346765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:35:42.346765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:42.346765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:42.346765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 271/531] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:42.346765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fa4bf-31e8-45f3-b0f8-72414fda0ac0 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:42.361544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701cb2e-e32c-43e3-9647-896dee657f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:42.362191 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701cb2e-e32c-43e3-9647-896dee657f2b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:42.362585 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701cb2e-e32c-43e3-9647-896dee657f2b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:42.362741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701cb2e-e32c-43e3-9647-896dee657f2b None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:42.363376 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701cb2e-e32c-43e3-9647-896dee657f2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:42.363674 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701cb2e-e32c-43e3-9647-896dee657f2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:42.365538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701cb2e-e32c-43e3-9647-896dee657f2b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:42.378849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 261/532] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:42.379876 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d373dd4-c7c6-4710-ae97-431515dc20ee None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:42.380203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d373dd4-c7c6-4710-ae97-431515dc20ee None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:42.380463 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d373dd4-c7c6-4710-ae97-431515dc20ee None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:42.386232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 262/533] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:42.386232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d373dd4-c7c6-4710-ae97-431515dc20ee None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:42.475898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:42.507390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fa4bf-31e8-45f3-b0f8-72414fda0ac0 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:42.550467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c3b1a-d589-45d5-88ed-8527d615a1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:42.551210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c3b1a-d589-45d5-88ed-8527d615a1a5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:42.551487 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c3b1a-d589-45d5-88ed-8527d615a1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:42.551759 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c3b1a-d589-45d5-88ed-8527d615a1a5 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:42.552437 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c3b1a-d589-45d5-88ed-8527d615a1a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:42.552739 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:42.553421 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c3b1a-d589-45d5-88ed-8527d615a1a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:42.555012 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c3b1a-d589-45d5-88ed-8527d615a1a5 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:42.559807 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d373dd4-c7c6-4710-ae97-431515dc20ee None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:42.577222 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 272/534] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:42.577222 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c3b1a-d589-45d5-88ed-8527d615a1a5 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:42.623657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e48879-db8c-46b7-ae23-a40e77dac195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:42.624823 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e48879-db8c-46b7-ae23-a40e77dac195 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:42.625289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e48879-db8c-46b7-ae23-a40e77dac195 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:42.625747 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e48879-db8c-46b7-ae23-a40e77dac195 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:42.627002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e48879-db8c-46b7-ae23-a40e77dac195 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:42.627515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e48879-db8c-46b7-ae23-a40e77dac195 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:42.629900 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e48879-db8c-46b7-ae23-a40e77dac195 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:42.651708 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 263/535] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:35:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:42.653438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699a63e-658a-49f9-bdf7-f06ad6c2f223 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:42.653720 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699a63e-658a-49f9-bdf7-f06ad6c2f223 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:42.653999 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699a63e-658a-49f9-bdf7-f06ad6c2f223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:42.674822 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6699a63e-658a-49f9-bdf7-f06ad6c2f223 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:42.676065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6699a63e-658a-49f9-bdf7-f06ad6c2f223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:42.757829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 264/536] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:42.758799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699a63e-658a-49f9-bdf7-f06ad6c2f223 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:42.781389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c3b1a-d589-45d5-88ed-8527d615a1a5 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:42.834387 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa4462-58a6-4338-9026-89607266d6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:42.834977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa4462-58a6-4338-9026-89607266d6dc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:42.835259 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa4462-58a6-4338-9026-89607266d6dc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:42.835539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa4462-58a6-4338-9026-89607266d6dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:42.840583 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:42.848106 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa4462-58a6-4338-9026-89607266d6dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:42.850189 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa4462-58a6-4338-9026-89607266d6dc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-9faa4462-58a6-4338-9026-89607266d6dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:35:42.867129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:42.869196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:42.871105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:35:42.871105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:35:42.871105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:35:42.871105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:35:42.871105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:35:42.871105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:35:42.871105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:42.871105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:42.871105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 273/537] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:42.871105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa4462-58a6-4338-9026-89607266d6dc None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:42.925434 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:42.937059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699a63e-658a-49f9-bdf7-f06ad6c2f223 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:42.976091 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbdcae-67a3-4182-b143-60397980bb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:42.976842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbdcae-67a3-4182-b143-60397980bb6f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:42.977192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbdcae-67a3-4182-b143-60397980bb6f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:42.977542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbdcae-67a3-4182-b143-60397980bb6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:42.986148 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbdcae-67a3-4182-b143-60397980bb6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:42.987833 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbdcae-67a3-4182-b143-60397980bb6f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-26bbdcae-67a3-4182-b143-60397980bb6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:35:43.001049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:43.002451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:43.004765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:35:43.004765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:35:43.004765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:35:43.004765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:35:43.004765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:35:43.004765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:35:43.004765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:43.004765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:43.004765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 265/538] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:43.004765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbdcae-67a3-4182-b143-60397980bb6f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:43.090622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa4462-58a6-4338-9026-89607266d6dc None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:43.149043 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbdcae-67a3-4182-b143-60397980bb6f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:43.156786 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9154a-c66a-492f-a677-3cc5451b8806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:43.157811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9154a-c66a-492f-a677-3cc5451b8806 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:43.158342 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9154a-c66a-492f-a677-3cc5451b8806 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:43.158818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9154a-c66a-492f-a677-3cc5451b8806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:43.160213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9154a-c66a-492f-a677-3cc5451b8806 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=02260480264b4f8f8a7e5cb647ac5d9e) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:43.160792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9154a-c66a-492f-a677-3cc5451b8806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:43.163637 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9154a-c66a-492f-a677-3cc5451b8806 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:43.174169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 274/539] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:35:42 2022] GET /identity/v3/role_assignments?user.id=02260480264b4f8f8a7e5cb647ac5d9e&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:43.177427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9154a-c66a-492f-a677-3cc5451b8806 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:43.193077 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9cf8f-d959-4016-a436-99a14dd2882d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:43.193807 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9cf8f-d959-4016-a436-99a14dd2882d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:43.194190 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9cf8f-d959-4016-a436-99a14dd2882d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:43.194675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9cf8f-d959-4016-a436-99a14dd2882d None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:43.195522 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9cf8f-d959-4016-a436-99a14dd2882d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:43.195901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9cf8f-d959-4016-a436-99a14dd2882d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:43.197493 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9cf8f-d959-4016-a436-99a14dd2882d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:43.215605 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 266/540] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:35:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:43.217465 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe1838-a910-4cd4-baa5-75f6befc7649 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:43.217802 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe1838-a910-4cd4-baa5-75f6befc7649 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:43.218169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe1838-a910-4cd4-baa5-75f6befc7649 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:43.220529 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 267/541] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:43.225396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe1838-a910-4cd4-baa5-75f6befc7649 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:43.243099 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:43.306127 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:43.365849 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9154a-c66a-492f-a677-3cc5451b8806 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:43.398183 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe1838-a910-4cd4-baa5-75f6befc7649 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:43.409742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112295fc-54b0-4ffd-8b3c-22df910d6b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:43.410399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112295fc-54b0-4ffd-8b3c-22df910d6b3d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:43.410673 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112295fc-54b0-4ffd-8b3c-22df910d6b3d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:43.410940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112295fc-54b0-4ffd-8b3c-22df910d6b3d None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:43.411642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112295fc-54b0-4ffd-8b3c-22df910d6b3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:43.411943 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112295fc-54b0-4ffd-8b3c-22df910d6b3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:43.413453 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112295fc-54b0-4ffd-8b3c-22df910d6b3d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:43.438512 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 275/542] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:35:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:43.441181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad4f65-7cef-4e0a-a0d6-991f170af757 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:43.441604 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad4f65-7cef-4e0a-a0d6-991f170af757 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:43.442286 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad4f65-7cef-4e0a-a0d6-991f170af757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:43.462059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-62ad4f65-7cef-4e0a-a0d6-991f170af757 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:43.463451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-62ad4f65-7cef-4e0a-a0d6-991f170af757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:43.464903 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727f737-2a88-4109-aaa1-9b0f8db6a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:43.465883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727f737-2a88-4109-aaa1-9b0f8db6a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:43.466361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727f737-2a88-4109-aaa1-9b0f8db6a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:43.466785 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727f737-2a88-4109-aaa1-9b0f8db6a5c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:43.468043 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727f737-2a88-4109-aaa1-9b0f8db6a5c4 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=b131edeaa17e4e10af385339d463a512) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:43.468549 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727f737-2a88-4109-aaa1-9b0f8db6a5c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:43.471344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727f737-2a88-4109-aaa1-9b0f8db6a5c4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:43.482074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 268/543] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:35:43 2022] GET /identity/v3/role_assignments?user.id=b131edeaa17e4e10af385339d463a512&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:43.483320 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727f737-2a88-4109-aaa1-9b0f8db6a5c4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:43.600168 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 276/544] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:43.601854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad4f65-7cef-4e0a-a0d6-991f170af757 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:43.616541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:43.695180 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727f737-2a88-4109-aaa1-9b0f8db6a5c4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:43.762299 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c987c-a924-4432-b3f0-43a3c4c48de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:43.763646 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c987c-a924-4432-b3f0-43a3c4c48de4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:43.764360 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c987c-a924-4432-b3f0-43a3c4c48de4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:43.765007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c987c-a924-4432-b3f0-43a3c4c48de4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:43.768080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:43.789987 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c987c-a924-4432-b3f0-43a3c4c48de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-274c987c-a924-4432-b3f0-43a3c4c48de4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:43.802751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:43.804710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:43.806567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:43.806567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:43.806567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:43.806567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:43.806567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:43.806567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:43.806567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:43.806567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:43.806567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:43.806567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:43.806567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:43.806567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:43.811106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 269/545] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:43.821897 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c987c-a924-4432-b3f0-43a3c4c48de4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:43.861545 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad4f65-7cef-4e0a-a0d6-991f170af757 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:43.943147 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742253e0-8dd6-473e-acc9-27e520717f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:43.944176 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742253e0-8dd6-473e-acc9-27e520717f17 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:43.944462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742253e0-8dd6-473e-acc9-27e520717f17 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:43.945010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742253e0-8dd6-473e-acc9-27e520717f17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:43.945765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742253e0-8dd6-473e-acc9-27e520717f17 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=756aec42473f4bebb15f8d43fe16504d) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:43.946101 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742253e0-8dd6-473e-acc9-27e520717f17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:43.948439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742253e0-8dd6-473e-acc9-27e520717f17 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:43.957222 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 277/546] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:35:43 2022] GET /identity/v3/role_assignments?user.id=756aec42473f4bebb15f8d43fe16504d&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 259 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:43.958251 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742253e0-8dd6-473e-acc9-27e520717f17 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:44.044031 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c987c-a924-4432-b3f0-43a3c4c48de4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:44.099813 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3580a00-1c46-44ae-97f9-c08c8a9adad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:44.100445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3580a00-1c46-44ae-97f9-c08c8a9adad0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:44.100723 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3580a00-1c46-44ae-97f9-c08c8a9adad0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:44.100989 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3580a00-1c46-44ae-97f9-c08c8a9adad0 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:44.101674 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3580a00-1c46-44ae-97f9-c08c8a9adad0 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:44.101982 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3580a00-1c46-44ae-97f9-c08c8a9adad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:44.103443 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3580a00-1c46-44ae-97f9-c08c8a9adad0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:44.105667 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:44.122339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 270/547] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:44.132441 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3580a00-1c46-44ae-97f9-c08c8a9adad0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:44.150012 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742253e0-8dd6-473e-acc9-27e520717f17 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:44.186667 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:44.224156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc98fa-11ae-43c2-9fae-0e60c4a1aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:44.224716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc98fa-11ae-43c2-9fae-0e60c4a1aa5a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:44.224988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc98fa-11ae-43c2-9fae-0e60c4a1aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:44.225260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc98fa-11ae-43c2-9fae-0e60c4a1aa5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:44.250769 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc98fa-11ae-43c2-9fae-0e60c4a1aa5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-5edc98fa-11ae-43c2-9fae-0e60c4a1aa5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:44.257743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:44.259536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:44.261711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 278/548] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:44.265642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc98fa-11ae-43c2-9fae-0e60c4a1aa5a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:44.319528 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3580a00-1c46-44ae-97f9-c08c8a9adad0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:44.391136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc98fa-11ae-43c2-9fae-0e60c4a1aa5a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:44.434343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12afcab-1117-4530-866f-12befb34e87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:44.434975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12afcab-1117-4530-866f-12befb34e87c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:44.435254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12afcab-1117-4530-866f-12befb34e87c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:44.435513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12afcab-1117-4530-866f-12befb34e87c None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:44.436684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12afcab-1117-4530-866f-12befb34e87c None admin] RBAC: Adding query filter params (name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:44.436988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12afcab-1117-4530-866f-12befb34e87c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:44.438444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12afcab-1117-4530-866f-12befb34e87c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:44.441459 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2836b6-31fe-4a56-b3e5-9851dc4272a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:44.442134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2836b6-31fe-4a56-b3e5-9851dc4272a2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:44.442410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2836b6-31fe-4a56-b3e5-9851dc4272a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:44.442677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2836b6-31fe-4a56-b3e5-9851dc4272a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:44.461289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 279/549] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:44.467177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2836b6-31fe-4a56-b3e5-9851dc4272a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:44.468678 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2836b6-31fe-4a56-b3e5-9851dc4272a2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:44.481440 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12afcab-1117-4530-866f-12befb34e87c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-da2836b6-31fe-4a56-b3e5-9851dc4272a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:44.518483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:44.520777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:44.522705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:44.522705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:44.522705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:44.528056 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 271/550] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:44.530172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:44.536967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2836b6-31fe-4a56-b3e5-9851dc4272a2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:44.631601 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:44.663796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12afcab-1117-4530-866f-12befb34e87c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:44.731072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2836b6-31fe-4a56-b3e5-9851dc4272a2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:44.733711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f4311-b082-4866-af7a-6e9c53022d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:44.734357 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f4311-b082-4866-af7a-6e9c53022d91 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:44.734635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f4311-b082-4866-af7a-6e9c53022d91 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:44.734905 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f4311-b082-4866-af7a-6e9c53022d91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:44.758624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f4311-b082-4866-af7a-6e9c53022d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:44.761216 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f4311-b082-4866-af7a-6e9c53022d91 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-886f4311-b082-4866-af7a-6e9c53022d91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:44.795164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:44.796144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:44.797000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:44.797000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:44.797000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:44.800052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 280/551] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:44.806014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f4311-b082-4866-af7a-6e9c53022d91 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:44.810866 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed48c0-25c0-4683-ade4-141b48d3efe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:44.813423 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed48c0-25c0-4683-ade4-141b48d3efe3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:44.814396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed48c0-25c0-4683-ade4-141b48d3efe3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:44.814895 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed48c0-25c0-4683-ade4-141b48d3efe3 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:44.818545 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed48c0-25c0-4683-ade4-141b48d3efe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:44.819317 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed48c0-25c0-4683-ade4-141b48d3efe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:44.821087 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed48c0-25c0-4683-ade4-141b48d3efe3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:44.849788 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 272/552] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:44.856630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed48c0-25c0-4683-ade4-141b48d3efe3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:44.981021 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed48c0-25c0-4683-ade4-141b48d3efe3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:44.998774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:45.024282 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0bcca-9bae-4bb6-a2aa-3f245751e311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:45.024938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0bcca-9bae-4bb6-a2aa-3f245751e311 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:45.025217 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0bcca-9bae-4bb6-a2aa-3f245751e311 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:45.025468 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0bcca-9bae-4bb6-a2aa-3f245751e311 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:45.033660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0bcca-9bae-4bb6-a2aa-3f245751e311 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:45.036006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0bcca-9bae-4bb6-a2aa-3f245751e311 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-34a0bcca-9bae-4bb6-a2aa-3f245751e311 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:45.050359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:45.052912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:45.054668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:45.054668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:45.054668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:45.054668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:45.054668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:45.054668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:45.054668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:45.054668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:45.054668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 12:35:45.054668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:45.054668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 273/553] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:35:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:45.061536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0bcca-9bae-4bb6-a2aa-3f245751e311 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:45.077877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:45.124337 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f4311-b082-4866-af7a-6e9c53022d91 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:45.187949 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0bcca-9bae-4bb6-a2aa-3f245751e311 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:45.229716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6fd97-36b4-4c77-a8ca-0cdcd8937933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:45.231627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b246f1-7b0a-45a4-96f6-3a1fe6548ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:45.231944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6fd97-36b4-4c77-a8ca-0cdcd8937933 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:45.232451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b246f1-7b0a-45a4-96f6-3a1fe6548ecd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:45.232831 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b246f1-7b0a-45a4-96f6-3a1fe6548ecd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:45.233056 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6fd97-36b4-4c77-a8ca-0cdcd8937933 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:45.233288 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b246f1-7b0a-45a4-96f6-3a1fe6548ecd None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:45.233918 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6fd97-36b4-4c77-a8ca-0cdcd8937933 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:45.234346 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b246f1-7b0a-45a4-96f6-3a1fe6548ecd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:45.234767 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b246f1-7b0a-45a4-96f6-3a1fe6548ecd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:45.242082 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6fd97-36b4-4c77-a8ca-0cdcd8937933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:45.242482 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b246f1-7b0a-45a4-96f6-3a1fe6548ecd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:45.243441 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6fd97-36b4-4c77-a8ca-0cdcd8937933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:45.247008 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6fd97-36b4-4c77-a8ca-0cdcd8937933 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:45.259535 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 274/554] 10.222.0.33 () {56 vars in 1135 bytes} [Tue Oct 4 12:35:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:45.265388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b246f1-7b0a-45a4-96f6-3a1fe6548ecd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:45.298188 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:45.306010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 281/555] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:45.321923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6fd97-36b4-4c77-a8ca-0cdcd8937933 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:45.393138 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b246f1-7b0a-45a4-96f6-3a1fe6548ecd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:45.434250 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb09acc-da30-4746-9d5d-d226ab69d643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:45.434990 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb09acc-da30-4746-9d5d-d226ab69d643 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:45.435328 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb09acc-da30-4746-9d5d-d226ab69d643 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:45.435663 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb09acc-da30-4746-9d5d-d226ab69d643 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:45.512552 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb09acc-da30-4746-9d5d-d226ab69d643 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:45.514028 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb09acc-da30-4746-9d5d-d226ab69d643 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:45.544191 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:45.627690 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 275/556] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:35:45.633447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb09acc-da30-4746-9d5d-d226ab69d643 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:45.725791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6fd97-36b4-4c77-a8ca-0cdcd8937933 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:45.752067 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb09acc-da30-4746-9d5d-d226ab69d643 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:45.782158 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:45.793830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854489c5-cfff-464b-b8ca-70faae2c22f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:45.794869 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854489c5-cfff-464b-b8ca-70faae2c22f0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:45.795348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854489c5-cfff-464b-b8ca-70faae2c22f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:45.795819 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854489c5-cfff-464b-b8ca-70faae2c22f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:45.808956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854489c5-cfff-464b-b8ca-70faae2c22f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:45.810853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854489c5-cfff-464b-b8ca-70faae2c22f0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:45.816573 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a2a3b-2a7c-429f-be27-c96e9be21492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:45.817226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a2a3b-2a7c-429f-be27-c96e9be21492 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:45.817756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a2a3b-2a7c-429f-be27-c96e9be21492 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:45.817756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a2a3b-2a7c-429f-be27-c96e9be21492 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:45.830848 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a2a3b-2a7c-429f-be27-c96e9be21492 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:45.831849 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a2a3b-2a7c-429f-be27-c96e9be21492 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-854489c5-cfff-464b-b8ca-70faae2c22f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:45.837935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:45.839497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:45.840355 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:45.840355 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:45.840355 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-243a2a3b-2a7c-429f-be27-c96e9be21492 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:45.846065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:45.847040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:45.848047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:45.848047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:45.848047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:45.848047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:45.848047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:45.848047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:45.848047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:45.848047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:45.848047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 12:35:45.848047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:45.848047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 282/557] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 526 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:45.849555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 276/558] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:45.852069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a2a3b-2a7c-429f-be27-c96e9be21492 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:45.856141 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854489c5-cfff-464b-b8ca-70faae2c22f0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:45.878129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:46.059079 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854489c5-cfff-464b-b8ca-70faae2c22f0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:46.092368 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a2a3b-2a7c-429f-be27-c96e9be21492 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:46.100972 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b017234-a702-4cc3-945c-c7e92d068cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:46.101688 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b017234-a702-4cc3-945c-c7e92d068cb1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:46.102049 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b017234-a702-4cc3-945c-c7e92d068cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:46.102401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b017234-a702-4cc3-945c-c7e92d068cb1 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:46.103122 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b017234-a702-4cc3-945c-c7e92d068cb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:46.103490 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b017234-a702-4cc3-945c-c7e92d068cb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:46.105713 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b017234-a702-4cc3-945c-c7e92d068cb1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:46.120480 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 277/559] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:46.126448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b017234-a702-4cc3-945c-c7e92d068cb1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:46.161287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea3062-2b83-44f7-a7e4-ac272899d6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:46.162568 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea3062-2b83-44f7-a7e4-ac272899d6ba None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:46.162843 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea3062-2b83-44f7-a7e4-ac272899d6ba None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:46.163112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea3062-2b83-44f7-a7e4-ac272899d6ba None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:46.164242 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea3062-2b83-44f7-a7e4-ac272899d6ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:46.164554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea3062-2b83-44f7-a7e4-ac272899d6ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:46.166101 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea3062-2b83-44f7-a7e4-ac272899d6ba None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:46.189977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 283/560] 10.222.0.33 () {56 vars in 1137 bytes} [Tue Oct 4 12:35:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:46.196507 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cea9b1-463e-437d-9c3a-b881b4dc044b None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:46.197009 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cea9b1-463e-437d-9c3a-b881b4dc044b None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:46.198059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cea9b1-463e-437d-9c3a-b881b4dc044b None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:46.200718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 284/561] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:46.205866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cea9b1-463e-437d-9c3a-b881b4dc044b None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:46.224985 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:46.287138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:46.332024 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cea9b1-463e-437d-9c3a-b881b4dc044b None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:46.351964 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b017234-a702-4cc3-945c-c7e92d068cb1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:46.375598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bef6c8-9b2e-4e22-9130-01e4644c3a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:46.376682 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bef6c8-9b2e-4e22-9130-01e4644c3a87 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:46.377164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bef6c8-9b2e-4e22-9130-01e4644c3a87 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:46.377658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bef6c8-9b2e-4e22-9130-01e4644c3a87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:46.390602 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bef6c8-9b2e-4e22-9130-01e4644c3a87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:46.392497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bef6c8-9b2e-4e22-9130-01e4644c3a87 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-e0bef6c8-9b2e-4e22-9130-01e4644c3a87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:46.409550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:46.410811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:46.411706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:46.411706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:46.411706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:46.418216 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 285/562] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:46.419382 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc53c77-622e-435c-8138-831b82ad8013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:46.420304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1dc9b-f9bc-457b-8220-4627e90b1244 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:46.420420 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc53c77-622e-435c-8138-831b82ad8013 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:46.420743 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc53c77-622e-435c-8138-831b82ad8013 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:46.421127 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1dc9b-f9bc-457b-8220-4627e90b1244 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:46.421231 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc53c77-622e-435c-8138-831b82ad8013 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:46.421683 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1dc9b-f9bc-457b-8220-4627e90b1244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:46.434879 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc53c77-622e-435c-8138-831b82ad8013 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:46.437818 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc53c77-622e-435c-8138-831b82ad8013 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:46.445390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-51c1dc9b-f9bc-457b-8220-4627e90b1244 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:46.447140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-51c1dc9b-f9bc-457b-8220-4627e90b1244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-5cc53c77-622e-435c-8138-831b82ad8013 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:35:46.461232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:46.462229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:46.463159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:35:46.463159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:35:46.463159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:35:46.463159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:35:46.463159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:35:46.463159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:35:46.463159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:46.463159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:46.463159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 278/563] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:46.464688 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc53c77-622e-435c-8138-831b82ad8013 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:46.529649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 286/564] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:46.532730 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ceab0-fc1a-47fe-b749-75409eaa4e30 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:46.533267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ceab0-fc1a-47fe-b749-75409eaa4e30 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:46.533734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ceab0-fc1a-47fe-b749-75409eaa4e30 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:46.535724 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 287/565] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:46.536761 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ceab0-fc1a-47fe-b749-75409eaa4e30 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:46.615749 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:46.621092 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc53c77-622e-435c-8138-831b82ad8013 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:46.638546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:46.667325 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e6e49-ee44-458d-a147-20c0196f6763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:46.668369 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e6e49-ee44-458d-a147-20c0196f6763 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:46.668845 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e6e49-ee44-458d-a147-20c0196f6763 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:46.669358 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e6e49-ee44-458d-a147-20c0196f6763 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:46.670331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e6e49-ee44-458d-a147-20c0196f6763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:46.670848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e6e49-ee44-458d-a147-20c0196f6763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:46.672633 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e6e49-ee44-458d-a147-20c0196f6763 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:46.692167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 279/566] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:46.694525 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c51db-f966-4108-9e18-2dc431585ad7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:46.695023 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c51db-f966-4108-9e18-2dc431585ad7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:46.695512 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c51db-f966-4108-9e18-2dc431585ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:46.727812 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ceab0-fc1a-47fe-b749-75409eaa4e30 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:46.730716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-639c51db-f966-4108-9e18-2dc431585ad7 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:46.732739 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-639c51db-f966-4108-9e18-2dc431585ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:46.777695 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac11b7-54e7-4fa1-be57-50c078fc7b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:46.779137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac11b7-54e7-4fa1-be57-50c078fc7b64 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:46.779734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac11b7-54e7-4fa1-be57-50c078fc7b64 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:46.780268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac11b7-54e7-4fa1-be57-50c078fc7b64 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:46.781498 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac11b7-54e7-4fa1-be57-50c078fc7b64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:46.782061 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac11b7-54e7-4fa1-be57-50c078fc7b64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:46.784667 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac11b7-54e7-4fa1-be57-50c078fc7b64 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:46.801129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 288/567] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:35:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:46.802576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac11b7-54e7-4fa1-be57-50c078fc7b64 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:46.872977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 280/568] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:46.874191 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c51db-f966-4108-9e18-2dc431585ad7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:46.982620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:46.995610 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c51db-f966-4108-9e18-2dc431585ad7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:47.038153 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed8847-99ee-4955-952f-fc0092cd2b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:47.039284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed8847-99ee-4955-952f-fc0092cd2b94 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:47.039766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed8847-99ee-4955-952f-fc0092cd2b94 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:47.040251 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed8847-99ee-4955-952f-fc0092cd2b94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:47.041713 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed8847-99ee-4955-952f-fc0092cd2b94 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=332fdff1d02a4664923ddbe0478eaa7a) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:47.042249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed8847-99ee-4955-952f-fc0092cd2b94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:47.043931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed8847-99ee-4955-952f-fc0092cd2b94 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:47.051230 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 281/569] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:35:46 2022] GET /identity/v3/role_assignments?user.id=332fdff1d02a4664923ddbe0478eaa7a&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:47.079039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed8847-99ee-4955-952f-fc0092cd2b94 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:47.097115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac11b7-54e7-4fa1-be57-50c078fc7b64 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:47.118169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:47.176689 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e8552-12a1-4173-911d-815e0462461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:47.178390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e8552-12a1-4173-911d-815e0462461a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:47.179460 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e8552-12a1-4173-911d-815e0462461a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:47.180682 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e8552-12a1-4173-911d-815e0462461a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:47.196159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e8552-12a1-4173-911d-815e0462461a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:47.200429 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e8552-12a1-4173-911d-815e0462461a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-a79e8552-12a1-4173-911d-815e0462461a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:35:47.227444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:47.229415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:47.232347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:35:47.232347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:35:47.232347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:35:47.232347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:35:47.232347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:35:47.232347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:35:47.232347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:47.232347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:47.235642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 289/570] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:47.243741 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e8552-12a1-4173-911d-815e0462461a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:47.383344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed8847-99ee-4955-952f-fc0092cd2b94 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:47.425525 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00257c86-89b9-4df6-ae51-eb597ee9717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:47.426223 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00257c86-89b9-4df6-ae51-eb597ee9717a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:47.426500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00257c86-89b9-4df6-ae51-eb597ee9717a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:47.426836 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00257c86-89b9-4df6-ae51-eb597ee9717a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:47.440031 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00257c86-89b9-4df6-ae51-eb597ee9717a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-00257c86-89b9-4df6-ae51-eb597ee9717a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:47.445247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:47.447290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 282/571] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:47.449596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4ac31-9e03-443d-8ffd-5fe06527cf5f None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:47.451281 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4ac31-9e03-443d-8ffd-5fe06527cf5f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:47.451281 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4ac31-9e03-443d-8ffd-5fe06527cf5f None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:47.451281 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 283/572] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:47.455491 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bbe19-2314-4be1-905e-214bc76eb4f2 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:47.455926 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bbe19-2314-4be1-905e-214bc76eb4f2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:47.456308 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bbe19-2314-4be1-905e-214bc76eb4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:47.470163 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e8552-12a1-4173-911d-815e0462461a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:47.478176 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c15bbe19-2314-4be1-905e-214bc76eb4f2 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:47.480106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c15bbe19-2314-4be1-905e-214bc76eb4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:47.489249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:47.549274 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc296bb-0c57-4fad-bb41-3a469f675d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:47.550273 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc296bb-0c57-4fad-bb41-3a469f675d6a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:47.550762 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc296bb-0c57-4fad-bb41-3a469f675d6a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:47.551242 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc296bb-0c57-4fad-bb41-3a469f675d6a None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:47.552469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc296bb-0c57-4fad-bb41-3a469f675d6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:47.552896 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc296bb-0c57-4fad-bb41-3a469f675d6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:47.555701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc296bb-0c57-4fad-bb41-3a469f675d6a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:47.573763 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 284/573] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:47.573763 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bbe19-2314-4be1-905e-214bc76eb4f2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:47.583536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 290/574] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:35:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:47.583536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc296bb-0c57-4fad-bb41-3a469f675d6a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:47.636353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:47.707548 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bbe19-2314-4be1-905e-214bc76eb4f2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:47.751461 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62448b9-958f-41d4-a176-ca3a4872c55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:47.752455 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62448b9-958f-41d4-a176-ca3a4872c55e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:47.752943 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62448b9-958f-41d4-a176-ca3a4872c55e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:47.753460 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62448b9-958f-41d4-a176-ca3a4872c55e None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:47.754946 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62448b9-958f-41d4-a176-ca3a4872c55e None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:47.755258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62448b9-958f-41d4-a176-ca3a4872c55e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:47.756655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62448b9-958f-41d4-a176-ca3a4872c55e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:47.772792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 285/575] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:47.774351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62448b9-958f-41d4-a176-ca3a4872c55e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:47.782721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:47.784130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc296bb-0c57-4fad-bb41-3a469f675d6a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:47.872777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2d3a8-aff1-4ca6-9ade-21d9b9155455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:47.874769 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2d3a8-aff1-4ca6-9ade-21d9b9155455 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:47.875575 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2d3a8-aff1-4ca6-9ade-21d9b9155455 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:47.876769 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2d3a8-aff1-4ca6-9ade-21d9b9155455 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:47.903313 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2d3a8-aff1-4ca6-9ade-21d9b9155455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-abc2d3a8-aff1-4ca6-9ade-21d9b9155455 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:47.914647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:47.915643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:47.917090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:47.917090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:47.917090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:47.917090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:47.917090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:47.917090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:47.917090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:47.917090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:47.917090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:47.917090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:47.917090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:47.917090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:47.917703 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 291/576] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:47.923572 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2d3a8-aff1-4ca6-9ade-21d9b9155455 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:47.995593 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62448b9-958f-41d4-a176-ca3a4872c55e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:48.037647 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e98a83-3c6c-47b4-9570-9ad9d93b051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:48.038355 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e98a83-3c6c-47b4-9570-9ad9d93b051a None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:48.038639 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e98a83-3c6c-47b4-9570-9ad9d93b051a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:48.038904 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e98a83-3c6c-47b4-9570-9ad9d93b051a None admin] PATH_INFO: `/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/f2f4f9541f89433cad52e826794b049f/roles/943fc0edf23046a69ba6102f2c1e544c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:48.057385 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e98a83-3c6c-47b4-9570-9ad9d93b051a None admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fb003a695490faa0077bbbe5e7d77, user_id=f2f4f9541f89433cad52e826794b049f, role_id=943fc0edf23046a69ba6102f2c1e544c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:48.058902 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e98a83-3c6c-47b4-9570-9ad9d93b051a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:48.075408 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 286/577] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:35:47 2022] PUT /identity/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/f2f4f9541f89433cad52e826794b049f/roles/943fc0edf23046a69ba6102f2c1e544c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:35:48.077115 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e98a83-3c6c-47b4-9570-9ad9d93b051a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:48.097494 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:48.120313 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2d3a8-aff1-4ca6-9ade-21d9b9155455 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:48.174657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:48.188646 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e587f-b28e-4049-b08e-af980e0cdb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:48.189626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e587f-b28e-4049-b08e-af980e0cdb54 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:48.190029 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e587f-b28e-4049-b08e-af980e0cdb54 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:48.190543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e587f-b28e-4049-b08e-af980e0cdb54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:48.210862 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e587f-b28e-4049-b08e-af980e0cdb54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:48.213953 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e587f-b28e-4049-b08e-af980e0cdb54 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-1e8e587f-b28e-4049-b08e-af980e0cdb54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:48.247515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:48.248467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:48.249330 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:48.249330 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:48.249330 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:48.249917 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 292/578] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:48.251371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e587f-b28e-4049-b08e-af980e0cdb54 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:48.287972 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e98a83-3c6c-47b4-9570-9ad9d93b051a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:48.376728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97de2d-b1ac-4f80-8122-76e7f4907b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:48.378252 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97de2d-b1ac-4f80-8122-76e7f4907b5f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:48.378742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97de2d-b1ac-4f80-8122-76e7f4907b5f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:48.379226 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97de2d-b1ac-4f80-8122-76e7f4907b5f None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:48.381437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97de2d-b1ac-4f80-8122-76e7f4907b5f None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:48.381825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97de2d-b1ac-4f80-8122-76e7f4907b5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:48.386296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97de2d-b1ac-4f80-8122-76e7f4907b5f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:48.418111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 287/579] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:48.418111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97de2d-b1ac-4f80-8122-76e7f4907b5f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:48.432296 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e587f-b28e-4049-b08e-af980e0cdb54 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:48.483054 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf7e29-c6d1-47db-85ca-7db01f01be6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:48.484079 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf7e29-c6d1-47db-85ca-7db01f01be6a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:48.490071 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf7e29-c6d1-47db-85ca-7db01f01be6a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:48.490071 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf7e29-c6d1-47db-85ca-7db01f01be6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:48.496889 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:48.503420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf7e29-c6d1-47db-85ca-7db01f01be6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-3cdf7e29-c6d1-47db-85ca-7db01f01be6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:48.510796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:48.512551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 293/580] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:48.514888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf7e29-c6d1-47db-85ca-7db01f01be6a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:48.587045 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97de2d-b1ac-4f80-8122-76e7f4907b5f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:48.621213 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:48.649714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf7e29-c6d1-47db-85ca-7db01f01be6a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:48.659677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052fd60-2651-4fad-96b8-37f0967be855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:48.661367 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052fd60-2651-4fad-96b8-37f0967be855 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:48.662356 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052fd60-2651-4fad-96b8-37f0967be855 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:48.663175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052fd60-2651-4fad-96b8-37f0967be855 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:48.664996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052fd60-2651-4fad-96b8-37f0967be855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:48.666019 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052fd60-2651-4fad-96b8-37f0967be855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:48.669264 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052fd60-2651-4fad-96b8-37f0967be855 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:48.690009 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c6af9-2f86-498f-a1dc-2a20b2531fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:48.690698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c6af9-2f86-498f-a1dc-2a20b2531fcf None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:48.690969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c6af9-2f86-498f-a1dc-2a20b2531fcf None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:48.691236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c6af9-2f86-498f-a1dc-2a20b2531fcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:48.697151 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 288/581] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:48.700139 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052fd60-2651-4fad-96b8-37f0967be855 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:48.705604 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c6af9-2f86-498f-a1dc-2a20b2531fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:48.707028 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c6af9-2f86-498f-a1dc-2a20b2531fcf None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-364c6af9-2f86-498f-a1dc-2a20b2531fcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:48.723175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:48.725084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:48.726856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:48.726856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:48.726856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:48.726856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 294/582] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:48.726856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c6af9-2f86-498f-a1dc-2a20b2531fcf None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:48.788017 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:48.848461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c6af9-2f86-498f-a1dc-2a20b2531fcf None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:48.890820 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de981e-3cd0-40cc-aef2-1fdf8264d0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:48.891470 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de981e-3cd0-40cc-aef2-1fdf8264d0b4 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:48.891750 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de981e-3cd0-40cc-aef2-1fdf8264d0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:48.892047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de981e-3cd0-40cc-aef2-1fdf8264d0b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:48.901649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de981e-3cd0-40cc-aef2-1fdf8264d0b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:48.903299 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de981e-3cd0-40cc-aef2-1fdf8264d0b4 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:48.910685 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052fd60-2651-4fad-96b8-37f0967be855 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-47de981e-3cd0-40cc-aef2-1fdf8264d0b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:48.916836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:48.918622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:48.920343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:48.920343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:48.920343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:48.920343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:48.920343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:48.920343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:48.920343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:48.920343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:48.920343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:35:48.920343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:48.920343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 295/583] 10.222.0.33 () {56 vars in 1093 bytes} [Tue Oct 4 12:35:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:48.920343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de981e-3cd0-40cc-aef2-1fdf8264d0b4 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:48.986163 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022e7cc-8ff6-4fc7-b43a-cd7290d24c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:48.987741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022e7cc-8ff6-4fc7-b43a-cd7290d24c71 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:48.988510 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022e7cc-8ff6-4fc7-b43a-cd7290d24c71 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:48.989232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9022e7cc-8ff6-4fc7-b43a-cd7290d24c71 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:48.991013 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022e7cc-8ff6-4fc7-b43a-cd7290d24c71 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:48.991908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022e7cc-8ff6-4fc7-b43a-cd7290d24c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:48.995886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9022e7cc-8ff6-4fc7-b43a-cd7290d24c71 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:49.018251 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 289/584] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:49.020479 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022e7cc-8ff6-4fc7-b43a-cd7290d24c71 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:49.050710 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de981e-3cd0-40cc-aef2-1fdf8264d0b4 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:49.068078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:49.094628 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef3084-f33e-4461-967c-71facfdd1e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:49.095255 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef3084-f33e-4461-967c-71facfdd1e0f None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:49.095536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef3084-f33e-4461-967c-71facfdd1e0f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:49.095834 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef3084-f33e-4461-967c-71facfdd1e0f None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:49.096543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef3084-f33e-4461-967c-71facfdd1e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:49.096852 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef3084-f33e-4461-967c-71facfdd1e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:49.098310 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef3084-f33e-4461-967c-71facfdd1e0f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:49.111708 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:49.114493 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 296/585] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:49.115588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef3084-f33e-4461-967c-71facfdd1e0f None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:49.234944 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9022e7cc-8ff6-4fc7-b43a-cd7290d24c71 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:49.308732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a366ef5-7114-4bd9-9e87-1f998ac978be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:49.309909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a366ef5-7114-4bd9-9e87-1f998ac978be None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:49.310572 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a366ef5-7114-4bd9-9e87-1f998ac978be None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:49.311157 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a366ef5-7114-4bd9-9e87-1f998ac978be None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:49.312569 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a366ef5-7114-4bd9-9e87-1f998ac978be None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:49.313205 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a366ef5-7114-4bd9-9e87-1f998ac978be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:49.314074 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef3084-f33e-4461-967c-71facfdd1e0f None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:49.315999 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a366ef5-7114-4bd9-9e87-1f998ac978be None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:49.350219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 290/586] 10.222.0.33 () {56 vars in 1133 bytes} [Tue Oct 4 12:35:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:49.351964 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a366ef5-7114-4bd9-9e87-1f998ac978be None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:49.385568 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28857e8c-5223-468b-85f0-11e71f3b9bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:49.387212 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28857e8c-5223-468b-85f0-11e71f3b9bf7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:49.388029 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28857e8c-5223-468b-85f0-11e71f3b9bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:49.388830 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28857e8c-5223-468b-85f0-11e71f3b9bf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:49.410726 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28857e8c-5223-468b-85f0-11e71f3b9bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:49.414015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28857e8c-5223-468b-85f0-11e71f3b9bf7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-28857e8c-5223-468b-85f0-11e71f3b9bf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:49.444595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:49.446497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:49.448316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:49.448316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:49.448316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:49.448316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 297/587] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:49.448316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28857e8c-5223-468b-85f0-11e71f3b9bf7 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:49.498394 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:49.554502 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:49.565793 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a366ef5-7114-4bd9-9e87-1f998ac978be None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:49.625198 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bd2ce-b844-4157-8219-2e9886533100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:49.626287 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bd2ce-b844-4157-8219-2e9886533100 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:49.626766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bd2ce-b844-4157-8219-2e9886533100 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:49.627242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bd2ce-b844-4157-8219-2e9886533100 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:49.638079 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28857e8c-5223-468b-85f0-11e71f3b9bf7 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:49.640690 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bd2ce-b844-4157-8219-2e9886533100 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:49.643467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bd2ce-b844-4157-8219-2e9886533100 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-449bd2ce-b844-4157-8219-2e9886533100 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:49.667524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:49.669603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:49.671483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:49.671483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:49.671483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:49.671483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:49.671483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:49.671483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:49.671483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:49.671483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:49.671483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 12:35:49.671483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:49.671483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 291/588] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:49.671483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bd2ce-b844-4157-8219-2e9886533100 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:49.708796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246f0a7-b31e-45fd-9193-565adeb03723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:49.710203 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246f0a7-b31e-45fd-9193-565adeb03723 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:49.710887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246f0a7-b31e-45fd-9193-565adeb03723 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:49.711579 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246f0a7-b31e-45fd-9193-565adeb03723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:49.731952 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246f0a7-b31e-45fd-9193-565adeb03723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:49.733832 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246f0a7-b31e-45fd-9193-565adeb03723 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-b246f0a7-b31e-45fd-9193-565adeb03723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:49.756159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:49.757268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:49.758194 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:49.758194 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:49.758194 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:49.758194 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 298/589] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:49.760676 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047be28c-667a-40e5-8fad-aa30c3553c95 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:49.761175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047be28c-667a-40e5-8fad-aa30c3553c95 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:49.761643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047be28c-667a-40e5-8fad-aa30c3553c95 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:49.764182 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 299/590] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:49.767215 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047be28c-667a-40e5-8fad-aa30c3553c95 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:49.878192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bd2ce-b844-4157-8219-2e9886533100 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:49.890203 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:49.930655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8a67a-1dbf-43a6-ad09-023362f924ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:49.931305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8a67a-1dbf-43a6-ad09-023362f924ee None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:49.931624 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8a67a-1dbf-43a6-ad09-023362f924ee None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:49.931919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8a67a-1dbf-43a6-ad09-023362f924ee None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:49.932583 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8a67a-1dbf-43a6-ad09-023362f924ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:49.932888 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8a67a-1dbf-43a6-ad09-023362f924ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:49.934389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8a67a-1dbf-43a6-ad09-023362f924ee None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:49.935336 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047be28c-667a-40e5-8fad-aa30c3553c95 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:49.950095 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 292/591] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:49.950934 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8a67a-1dbf-43a6-ad09-023362f924ee None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:49.987387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:50.002717 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e35c3-6221-4b85-8337-2a2e428ab193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:50.003836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e35c3-6221-4b85-8337-2a2e428ab193 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:50.004204 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e35c3-6221-4b85-8337-2a2e428ab193 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:50.004551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e35c3-6221-4b85-8337-2a2e428ab193 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:50.005804 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e35c3-6221-4b85-8337-2a2e428ab193 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:50.006324 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e35c3-6221-4b85-8337-2a2e428ab193 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:50.008540 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e35c3-6221-4b85-8337-2a2e428ab193 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:50.046086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 300/592] 10.222.0.33 () {56 vars in 1137 bytes} [Tue Oct 4 12:35:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:50.046086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8f6fd-8023-4207-895d-19e34b7c7438 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:50.046086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8f6fd-8023-4207-895d-19e34b7c7438 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:50.046086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8f6fd-8023-4207-895d-19e34b7c7438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:50.076139 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-04c8f6fd-8023-4207-895d-19e34b7c7438 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:50.078164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-04c8f6fd-8023-4207-895d-19e34b7c7438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:50.150216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8a67a-1dbf-43a6-ad09-023362f924ee None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:50.202567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 301/593] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:50.203764 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfa979-a904-42c4-957b-27ff5de43d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:50.204017 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8f6fd-8023-4207-895d-19e34b7c7438 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:50.204777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfa979-a904-42c4-957b-27ff5de43d9b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:50.205247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfa979-a904-42c4-957b-27ff5de43d9b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:50.205718 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfa979-a904-42c4-957b-27ff5de43d9b None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:50.206923 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfa979-a904-42c4-957b-27ff5de43d9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:50.207443 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfa979-a904-42c4-957b-27ff5de43d9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:50.209929 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfa979-a904-42c4-957b-27ff5de43d9b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:50.233730 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 293/594] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:50.235351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfa979-a904-42c4-957b-27ff5de43d9b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:50.359174 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:50.391576 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:50.436029 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8f6fd-8023-4207-895d-19e34b7c7438 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:50.443388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfa979-a904-42c4-957b-27ff5de43d9b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:50.495649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f2a6a-9bfe-4867-a0bd-f8a61f22aaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:50.496844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f2a6a-9bfe-4867-a0bd-f8a61f22aaf4 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:50.497186 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f2a6a-9bfe-4867-a0bd-f8a61f22aaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:50.497653 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f2a6a-9bfe-4867-a0bd-f8a61f22aaf4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:50.511076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f2a6a-9bfe-4867-a0bd-f8a61f22aaf4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:50.513620 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f2a6a-9bfe-4867-a0bd-f8a61f22aaf4 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:50.530160 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d618f8-e1ed-4fa0-957d-94a094f92f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:50.531721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d618f8-e1ed-4fa0-957d-94a094f92f7e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:50.532466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d618f8-e1ed-4fa0-957d-94a094f92f7e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:50.533208 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d618f8-e1ed-4fa0-957d-94a094f92f7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-166f2a6a-9bfe-4867-a0bd-f8a61f22aaf4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:50.534774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:50.536473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:50.538354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:50.538354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:50.538354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:50.538354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:50.538354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:50.538354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:50.538354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:50.538354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:50.538354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 12:35:50.538354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:50.538354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 302/595] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:50.538354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f2a6a-9bfe-4867-a0bd-f8a61f22aaf4 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:50.552699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d618f8-e1ed-4fa0-957d-94a094f92f7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:50.554200 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d618f8-e1ed-4fa0-957d-94a094f92f7e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-41d618f8-e1ed-4fa0-957d-94a094f92f7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:50.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:50.573555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:50.575769 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:50.575769 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:50.575769 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:50.575769 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 294/596] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:50.575769 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d618f8-e1ed-4fa0-957d-94a094f92f7e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:50.728919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d618f8-e1ed-4fa0-957d-94a094f92f7e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:50.747303 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f2a6a-9bfe-4867-a0bd-f8a61f22aaf4 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:50.774094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:50.784043 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0283d90-7877-457e-b9fd-8f6c9859fe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:50.784665 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0283d90-7877-457e-b9fd-8f6c9859fe75 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:50.784941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0283d90-7877-457e-b9fd-8f6c9859fe75 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:50.785211 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0283d90-7877-457e-b9fd-8f6c9859fe75 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:50.785948 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0283d90-7877-457e-b9fd-8f6c9859fe75 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:50.786282 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0283d90-7877-457e-b9fd-8f6c9859fe75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:50.787888 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0283d90-7877-457e-b9fd-8f6c9859fe75 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:50.789486 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:50.808899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbcb27-394c-4609-91c5-8f9fbbee59ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:50.809549 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 295/597] 10.222.0.33 () {56 vars in 1139 bytes} [Tue Oct 4 12:35:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:50.809919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbcb27-394c-4609-91c5-8f9fbbee59ef None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:50.810427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbcb27-394c-4609-91c5-8f9fbbee59ef None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:50.810586 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0283d90-7877-457e-b9fd-8f6c9859fe75 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:50.810916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbcb27-394c-4609-91c5-8f9fbbee59ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:50.826759 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfbcb27-394c-4609-91c5-8f9fbbee59ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:50.829584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfbcb27-394c-4609-91c5-8f9fbbee59ef None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-ebfbcb27-394c-4609-91c5-8f9fbbee59ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:35:50.851475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:50.852431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:50.853307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:35:50.853307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:35:50.853307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:35:50.853307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:35:50.853307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:35:50.853307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:35:50.853307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:50.853307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:50.853777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 303/598] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:50.855063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbcb27-394c-4609-91c5-8f9fbbee59ef None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:50.960604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0283d90-7877-457e-b9fd-8f6c9859fe75 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:50.988737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbcb27-394c-4609-91c5-8f9fbbee59ef None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:51.023560 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19041a7-e8e9-4528-bdf0-7a1e0daf75c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:51.024611 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19041a7-e8e9-4528-bdf0-7a1e0daf75c5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:51.025087 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19041a7-e8e9-4528-bdf0-7a1e0daf75c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:51.025736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19041a7-e8e9-4528-bdf0-7a1e0daf75c5 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:51.026921 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19041a7-e8e9-4528-bdf0-7a1e0daf75c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:51.027431 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19041a7-e8e9-4528-bdf0-7a1e0daf75c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:51.028155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1669df-446a-4779-90ed-fa363358fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:51.028880 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1669df-446a-4779-90ed-fa363358fd63 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:51.029243 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1669df-446a-4779-90ed-fa363358fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:51.029608 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1669df-446a-4779-90ed-fa363358fd63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:51.030147 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19041a7-e8e9-4528-bdf0-7a1e0daf75c5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:51.041355 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1669df-446a-4779-90ed-fa363358fd63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-ad1669df-446a-4779-90ed-fa363358fd63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:51.046716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:51.048023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 304/599] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:51.049616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120385e-8cd1-4006-a44f-fad7b26d0dec None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:51.052314 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120385e-8cd1-4006-a44f-fad7b26d0dec None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:51.052314 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120385e-8cd1-4006-a44f-fad7b26d0dec None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:51.052839 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 296/600] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:51.054659 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19041a7-e8e9-4528-bdf0-7a1e0daf75c5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:51.055189 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 305/601] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:51.056215 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120385e-8cd1-4006-a44f-fad7b26d0dec None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:51.116485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:51.126919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:51.189960 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19041a7-e8e9-4528-bdf0-7a1e0daf75c5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:51.204536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120385e-8cd1-4006-a44f-fad7b26d0dec None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:51.241464 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cd55e-e430-4961-a5ac-b27b640afc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:51.242280 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cd55e-e430-4961-a5ac-b27b640afc2f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:51.242407 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cd55e-e430-4961-a5ac-b27b640afc2f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:51.242775 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cd55e-e430-4961-a5ac-b27b640afc2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:51.244186 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09f1d5-aa99-4e07-b482-bd0c5a103ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:51.244809 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09f1d5-aa99-4e07-b482-bd0c5a103ed3 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:51.245089 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09f1d5-aa99-4e07-b482-bd0c5a103ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:51.245358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09f1d5-aa99-4e07-b482-bd0c5a103ed3 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:51.246160 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09f1d5-aa99-4e07-b482-bd0c5a103ed3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:51.246457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09f1d5-aa99-4e07-b482-bd0c5a103ed3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:51.247865 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09f1d5-aa99-4e07-b482-bd0c5a103ed3 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:51.260302 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cd55e-e430-4961-a5ac-b27b640afc2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:51.262790 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cd55e-e430-4961-a5ac-b27b640afc2f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:51.268825 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 306/602] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:35:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:51.270332 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09f1d5-aa99-4e07-b482-bd0c5a103ed3 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-2e0cd55e-e430-4961-a5ac-b27b640afc2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:51.288660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:51.290681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:51.292344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:51.292344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:51.292344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:51.294220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 297/603] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:51.296056 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ec454-4978-46d9-b2fd-140e74a4b945 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:51.296450 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ec454-4978-46d9-b2fd-140e74a4b945 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:51.296829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ec454-4978-46d9-b2fd-140e74a4b945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:51.319321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e06ec454-4978-46d9-b2fd-140e74a4b945 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:51.320757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e06ec454-4978-46d9-b2fd-140e74a4b945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:51.404944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09f1d5-aa99-4e07-b482-bd0c5a103ed3 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:51.407604 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:51.409005 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 298/604] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:51.410802 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ec454-4978-46d9-b2fd-140e74a4b945 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:51.449411 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:51.469167 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d922a39-3c28-4014-a073-24736286309b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:51.469821 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d922a39-3c28-4014-a073-24736286309b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:51.470124 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d922a39-3c28-4014-a073-24736286309b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:51.470400 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d922a39-3c28-4014-a073-24736286309b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:51.477859 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d922a39-3c28-4014-a073-24736286309b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:51.479922 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d922a39-3c28-4014-a073-24736286309b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-5d922a39-3c28-4014-a073-24736286309b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:35:51.492900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:51.501132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:51.503091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:35:51.503091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:35:51.503091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:35:51.503091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:35:51.503091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:35:51.503091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:35:51.503091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:51.503091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:51.503091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 307/605] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:51.503091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d922a39-3c28-4014-a073-24736286309b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:51.565446 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ec454-4978-46d9-b2fd-140e74a4b945 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:51.608498 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0a510-0fa7-4597-98e6-e388c65884c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:51.609127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0a510-0fa7-4597-98e6-e388c65884c9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:51.609458 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0a510-0fa7-4597-98e6-e388c65884c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:51.609734 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0a510-0fa7-4597-98e6-e388c65884c9 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:51.610497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0a510-0fa7-4597-98e6-e388c65884c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:51.610808 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0a510-0fa7-4597-98e6-e388c65884c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:51.612212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0a510-0fa7-4597-98e6-e388c65884c9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:51.621514 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 299/606] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:51.622399 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0a510-0fa7-4597-98e6-e388c65884c9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:51.632812 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d922a39-3c28-4014-a073-24736286309b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:51.689659 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10d74a-738a-4fa2-bef7-4742ae551d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:51.690778 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d74a-738a-4fa2-bef7-4742ae551d8c None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:51.691261 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d74a-738a-4fa2-bef7-4742ae551d8c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:51.691747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d74a-738a-4fa2-bef7-4742ae551d8c None admin] PATH_INFO: `/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/02260480264b4f8f8a7e5cb647ac5d9e/roles/943fc0edf23046a69ba6102f2c1e544c` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:51.717532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10d74a-738a-4fa2-bef7-4742ae551d8c None admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fb003a695490faa0077bbbe5e7d77, user_id=02260480264b4f8f8a7e5cb647ac5d9e, role_id=943fc0edf23046a69ba6102f2c1e544c)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:51.720214 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10d74a-738a-4fa2-bef7-4742ae551d8c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:51.726267 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:51.750165 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 308/607] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:35:51 2022] PUT /identity/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/02260480264b4f8f8a7e5cb647ac5d9e/roles/943fc0edf23046a69ba6102f2c1e544c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:35:51.750165 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10d74a-738a-4fa2-bef7-4742ae551d8c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:51.750442 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0a510-0fa7-4597-98e6-e388c65884c9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:51.777198 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:51.811128 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0ffb7-bb8c-4abb-beab-3cdf55b04c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:51.812223 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0ffb7-bb8c-4abb-beab-3cdf55b04c5a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:51.812685 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0ffb7-bb8c-4abb-beab-3cdf55b04c5a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:51.813135 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0ffb7-bb8c-4abb-beab-3cdf55b04c5a None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:51.813992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0ffb7-bb8c-4abb-beab-3cdf55b04c5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:51.814524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0ffb7-bb8c-4abb-beab-3cdf55b04c5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:51.816161 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0ffb7-bb8c-4abb-beab-3cdf55b04c5a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:51.841072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 300/608] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:51.842689 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0ffb7-bb8c-4abb-beab-3cdf55b04c5a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:51.917492 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10d74a-738a-4fa2-bef7-4742ae551d8c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:51.982837 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b536a20-9645-45c3-8dd2-3891054716bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:51.983962 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b536a20-9645-45c3-8dd2-3891054716bd None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:51.984480 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b536a20-9645-45c3-8dd2-3891054716bd None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:51.984994 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b536a20-9645-45c3-8dd2-3891054716bd None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:51.986304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b536a20-9645-45c3-8dd2-3891054716bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:51.986880 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b536a20-9645-45c3-8dd2-3891054716bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:51.989787 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b536a20-9645-45c3-8dd2-3891054716bd None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:52.010466 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 309/609] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:35:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:52.012014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b536a20-9645-45c3-8dd2-3891054716bd None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:52.049484 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0ffb7-bb8c-4abb-beab-3cdf55b04c5a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:52.112172 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863909b-3265-4703-9842-6fd7271c9f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:52.112172 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863909b-3265-4703-9842-6fd7271c9f7a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:52.112626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863909b-3265-4703-9842-6fd7271c9f7a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:52.112951 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863909b-3265-4703-9842-6fd7271c9f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:52.126625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863909b-3265-4703-9842-6fd7271c9f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:52.128521 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863909b-3265-4703-9842-6fd7271c9f7a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-1863909b-3265-4703-9842-6fd7271c9f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:52.148627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:52.151063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:52.152787 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:52.152787 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:52.152787 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:52.157288 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 301/610] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:52.159376 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863909b-3265-4703-9842-6fd7271c9f7a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:52.160313 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:52.162972 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:52.208130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b536a20-9645-45c3-8dd2-3891054716bd None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:52.259443 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e463d-98cd-4b79-b865-698402371841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:52.261138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e463d-98cd-4b79-b865-698402371841 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:52.261539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e463d-98cd-4b79-b865-698402371841 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:52.262005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e463d-98cd-4b79-b865-698402371841 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:52.275223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e463d-98cd-4b79-b865-698402371841 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:52.278635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e463d-98cd-4b79-b865-698402371841 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-2e1e463d-98cd-4b79-b865-698402371841 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:35:52.300549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:52.302490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:52.304219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:35:52.304219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:35:52.304219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:35:52.304219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:35:52.304219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:35:52.304219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:35:52.304219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:52.304219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:52.304219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 310/611] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:52.304219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e463d-98cd-4b79-b865-698402371841 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:52.332531 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863909b-3265-4703-9842-6fd7271c9f7a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:52.390681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5e450-9af5-4476-8ff6-dad752ade0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:52.391550 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5e450-9af5-4476-8ff6-dad752ade0db None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:52.392006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5e450-9af5-4476-8ff6-dad752ade0db None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:52.392449 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5e450-9af5-4476-8ff6-dad752ade0db None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:52.409401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5e450-9af5-4476-8ff6-dad752ade0db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-95a5e450-9af5-4476-8ff6-dad752ade0db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:52.416770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:52.418687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 302/612] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:52.421101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5e450-9af5-4476-8ff6-dad752ade0db None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:52.492488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e463d-98cd-4b79-b865-698402371841 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:52.502198 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:52.556165 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054208b-643e-48d9-a9c4-e77933f76222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:52.557741 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054208b-643e-48d9-a9c4-e77933f76222 None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:52.558308 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054208b-643e-48d9-a9c4-e77933f76222 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:52.558795 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054208b-643e-48d9-a9c4-e77933f76222 None admin] PATH_INFO: `/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/b131edeaa17e4e10af385339d463a512/roles/943fc0edf23046a69ba6102f2c1e544c` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:52.560372 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5e450-9af5-4476-8ff6-dad752ade0db None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:52.564395 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:52.588866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054208b-643e-48d9-a9c4-e77933f76222 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fb003a695490faa0077bbbe5e7d77, user_id=b131edeaa17e4e10af385339d463a512, role_id=943fc0edf23046a69ba6102f2c1e544c)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:52.598935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054208b-643e-48d9-a9c4-e77933f76222 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:52.607190 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091f7ea-b6a1-4165-a8ec-f05d95e53dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:52.608061 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091f7ea-b6a1-4165-a8ec-f05d95e53dae None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:52.608902 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091f7ea-b6a1-4165-a8ec-f05d95e53dae None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:52.609330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091f7ea-b6a1-4165-a8ec-f05d95e53dae None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:52.610444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091f7ea-b6a1-4165-a8ec-f05d95e53dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:52.610967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091f7ea-b6a1-4165-a8ec-f05d95e53dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:52.613297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091f7ea-b6a1-4165-a8ec-f05d95e53dae None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:52.624500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 311/613] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:35:52 2022] PUT /identity/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/b131edeaa17e4e10af385339d463a512/roles/943fc0edf23046a69ba6102f2c1e544c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:35:52.626920 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054208b-643e-48d9-a9c4-e77933f76222 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:52.638281 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 303/614] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:52.641995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091f7ea-b6a1-4165-a8ec-f05d95e53dae None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:52.826745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091f7ea-b6a1-4165-a8ec-f05d95e53dae None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:52.859896 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054208b-643e-48d9-a9c4-e77933f76222 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:52.874862 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:52.897447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b18e4-d2e6-4bca-b19d-63ea3afbabee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:52.898922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b18e4-d2e6-4bca-b19d-63ea3afbabee None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:52.899647 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b18e4-d2e6-4bca-b19d-63ea3afbabee None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:52.900417 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b18e4-d2e6-4bca-b19d-63ea3afbabee None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:52.902405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b18e4-d2e6-4bca-b19d-63ea3afbabee None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:52.903201 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b18e4-d2e6-4bca-b19d-63ea3afbabee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:52.905746 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b18e4-d2e6-4bca-b19d-63ea3afbabee None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:52.922083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 304/615] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:52.924226 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b18e4-d2e6-4bca-b19d-63ea3afbabee None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:52.934440 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b96a0-e7cc-44a3-920c-5f5bbec36806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:52.935278 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b96a0-e7cc-44a3-920c-5f5bbec36806 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:52.935679 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b96a0-e7cc-44a3-920c-5f5bbec36806 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:52.936141 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b96a0-e7cc-44a3-920c-5f5bbec36806 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:52.937452 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b96a0-e7cc-44a3-920c-5f5bbec36806 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:52.937995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b96a0-e7cc-44a3-920c-5f5bbec36806 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:52.940900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b96a0-e7cc-44a3-920c-5f5bbec36806 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:52.965743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 312/616] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:35:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:52.967317 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b96a0-e7cc-44a3-920c-5f5bbec36806 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:53.062570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:53.131392 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b18e4-d2e6-4bca-b19d-63ea3afbabee None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:53.183720 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b96a0-e7cc-44a3-920c-5f5bbec36806 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:53.214295 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c18e7-d9ae-4e48-819e-b3d41382fc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:53.215487 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c18e7-d9ae-4e48-819e-b3d41382fc69 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:53.215914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c18e7-d9ae-4e48-819e-b3d41382fc69 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:53.216331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c18e7-d9ae-4e48-819e-b3d41382fc69 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:53.232632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c18e7-d9ae-4e48-819e-b3d41382fc69 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:53.235385 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c18e7-d9ae-4e48-819e-b3d41382fc69 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-ab4c18e7-d9ae-4e48-819e-b3d41382fc69 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:53.261853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:53.263584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:53.265274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:53.265274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:53.265274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:53.265274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:53.265274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:53.265274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:53.265274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:53.265274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:53.265274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 12:35:53.265274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:53.265274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 305/617] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:35:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:53.266368 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c18e7-d9ae-4e48-819e-b3d41382fc69 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:53.274098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6d4db-041c-496e-bed3-23224342a5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:53.275534 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6d4db-041c-496e-bed3-23224342a5d7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:53.276287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6d4db-041c-496e-bed3-23224342a5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:53.276988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6d4db-041c-496e-bed3-23224342a5d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:53.303069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6d4db-041c-496e-bed3-23224342a5d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:53.306242 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6d4db-041c-496e-bed3-23224342a5d7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-76c6d4db-041c-496e-bed3-23224342a5d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:53.332623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:53.333971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:53.336272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:53.336272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:53.336272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:53.346452 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 313/618] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:53.348486 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6d4db-041c-496e-bed3-23224342a5d7 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:53.364184 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:53.451354 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c18e7-d9ae-4e48-819e-b3d41382fc69 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:53.498163 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83103f90-738b-4cb9-9cf8-c13cd773f3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:53.498898 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83103f90-738b-4cb9-9cf8-c13cd773f3f2 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:53.499188 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83103f90-738b-4cb9-9cf8-c13cd773f3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:53.499446 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83103f90-738b-4cb9-9cf8-c13cd773f3f2 None admin] PATH_INFO: `/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/756aec42473f4bebb15f8d43fe16504d/roles/943fc0edf23046a69ba6102f2c1e544c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:53.531091 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83103f90-738b-4cb9-9cf8-c13cd773f3f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fb003a695490faa0077bbbe5e7d77, user_id=756aec42473f4bebb15f8d43fe16504d, role_id=943fc0edf23046a69ba6102f2c1e544c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:53.535757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83103f90-738b-4cb9-9cf8-c13cd773f3f2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:53.539097 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:53.559911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6d4db-041c-496e-bed3-23224342a5d7 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:53.565033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 306/619] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:35:53 2022] PUT /identity/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/756aec42473f4bebb15f8d43fe16504d/roles/943fc0edf23046a69ba6102f2c1e544c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:35:53.573003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83103f90-738b-4cb9-9cf8-c13cd773f3f2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:53.675514 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b38a8-71b2-47d6-845e-38d353359a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:53.676804 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b38a8-71b2-47d6-845e-38d353359a53 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:53.677415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b38a8-71b2-47d6-845e-38d353359a53 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:53.677993 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b38a8-71b2-47d6-845e-38d353359a53 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:53.679720 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b38a8-71b2-47d6-845e-38d353359a53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:53.680394 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b38a8-71b2-47d6-845e-38d353359a53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:53.684055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b38a8-71b2-47d6-845e-38d353359a53 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:53.734761 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 314/620] 10.222.0.33 () {56 vars in 1135 bytes} [Tue Oct 4 12:35:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:53.740774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b38a8-71b2-47d6-845e-38d353359a53 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:53.794664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:53.807480 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83103f90-738b-4cb9-9cf8-c13cd773f3f2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:53.846615 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952878ce-e8d1-45e0-acb7-78d4ed6c0aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:53.847235 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952878ce-e8d1-45e0-acb7-78d4ed6c0aa0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:53.847515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952878ce-e8d1-45e0-acb7-78d4ed6c0aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:53.847797 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952878ce-e8d1-45e0-acb7-78d4ed6c0aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:53.848452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952878ce-e8d1-45e0-acb7-78d4ed6c0aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:53.848761 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952878ce-e8d1-45e0-acb7-78d4ed6c0aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:53.850209 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952878ce-e8d1-45e0-acb7-78d4ed6c0aa0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:53.863814 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 307/621] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:53.875133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952878ce-e8d1-45e0-acb7-78d4ed6c0aa0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:53.971331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b38a8-71b2-47d6-845e-38d353359a53 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:54.045972 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d991ba8-d8b8-45ba-8de0-8e8737fa0024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:54.045972 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d991ba8-d8b8-45ba-8de0-8e8737fa0024 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:54.045972 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d991ba8-d8b8-45ba-8de0-8e8737fa0024 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:54.045972 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d991ba8-d8b8-45ba-8de0-8e8737fa0024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:54.066844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d991ba8-d8b8-45ba-8de0-8e8737fa0024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:54.069924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d991ba8-d8b8-45ba-8de0-8e8737fa0024 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-2d991ba8-d8b8-45ba-8de0-8e8737fa0024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:54.102995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:54.105312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:54.107698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:54.107698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:54.107698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:54.109489 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 315/622] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:54.118073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d991ba8-d8b8-45ba-8de0-8e8737fa0024 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:54.150952 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:54.208110 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952878ce-e8d1-45e0-acb7-78d4ed6c0aa0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:54.256450 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5981004-9c72-4b93-869e-ecd8b31ec1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:54.257107 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5981004-9c72-4b93-869e-ecd8b31ec1a4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:54.257376 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5981004-9c72-4b93-869e-ecd8b31ec1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:54.257645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5981004-9c72-4b93-869e-ecd8b31ec1a4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:54.272391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5981004-9c72-4b93-869e-ecd8b31ec1a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:54.274294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5981004-9c72-4b93-869e-ecd8b31ec1a4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-e5981004-9c72-4b93-869e-ecd8b31ec1a4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:54.290666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:54.291748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:54.292699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:54.292699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:54.292699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:54.292699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:54.292699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:54.292699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:54.292699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:54.292699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:54.292699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 12:35:54.292699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:54.296132 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 308/623] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:54.297939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21530ea1-4fb3-45e2-b7c5-6a603b933f37 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:54.298249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21530ea1-4fb3-45e2-b7c5-6a603b933f37 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:54.298528 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21530ea1-4fb3-45e2-b7c5-6a603b933f37 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:54.304178 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 309/624] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:54.306203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21530ea1-4fb3-45e2-b7c5-6a603b933f37 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:54.313273 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d991ba8-d8b8-45ba-8de0-8e8737fa0024 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:54.325877 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:54.369976 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d64802-4a8b-4db1-86d0-0d6876666a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:54.371265 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d64802-4a8b-4db1-86d0-0d6876666a4b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:54.372298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d64802-4a8b-4db1-86d0-0d6876666a4b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:54.372796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d64802-4a8b-4db1-86d0-0d6876666a4b None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:54.373676 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d64802-4a8b-4db1-86d0-0d6876666a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:54.374218 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d64802-4a8b-4db1-86d0-0d6876666a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:54.376617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d64802-4a8b-4db1-86d0-0d6876666a4b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:54.399483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 316/625] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:54.399827 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ffcf9-627a-4913-9ae4-845e45a2e1e8 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:54.400020 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ffcf9-627a-4913-9ae4-845e45a2e1e8 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:54.400412 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ffcf9-627a-4913-9ae4-845e45a2e1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:54.424153 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-be8ffcf9-627a-4913-9ae4-845e45a2e1e8 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:54.425385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-be8ffcf9-627a-4913-9ae4-845e45a2e1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:54.443118 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21530ea1-4fb3-45e2-b7c5-6a603b933f37 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:54.494150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9e4cd-7223-4680-8c14-f7fb4370c972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:54.494150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9e4cd-7223-4680-8c14-f7fb4370c972 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:54.494150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9e4cd-7223-4680-8c14-f7fb4370c972 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:54.494150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9e4cd-7223-4680-8c14-f7fb4370c972 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:54.495483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9e4cd-7223-4680-8c14-f7fb4370c972 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:54.496011 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9e4cd-7223-4680-8c14-f7fb4370c972 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:54.498631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9e4cd-7223-4680-8c14-f7fb4370c972 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:54.514329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:54.526095 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 317/626] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:54.528129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ffcf9-627a-4913-9ae4-845e45a2e1e8 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:54.535323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 310/627] 10.222.0.33 () {56 vars in 1137 bytes} [Tue Oct 4 12:35:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:54.549975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9e4cd-7223-4680-8c14-f7fb4370c972 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:54.648403 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ffcf9-627a-4913-9ae4-845e45a2e1e8 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:54.701289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d548cb8-c56c-4933-ba44-c86b0284bd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:54.702318 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d548cb8-c56c-4933-ba44-c86b0284bd3e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:54.702748 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d548cb8-c56c-4933-ba44-c86b0284bd3e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:54.703170 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d548cb8-c56c-4933-ba44-c86b0284bd3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:54.718533 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d548cb8-c56c-4933-ba44-c86b0284bd3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:54.720270 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d548cb8-c56c-4933-ba44-c86b0284bd3e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-7d548cb8-c56c-4933-ba44-c86b0284bd3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:35:54.734747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:54.735831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:54.736911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:35:54.736911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:35:54.736911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:35:54.736911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:35:54.736911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:35:54.736911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:35:54.736911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:54.736911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:54.739468 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 318/628] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:54.747970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d548cb8-c56c-4933-ba44-c86b0284bd3e None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:54.782622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:54.836461 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9e4cd-7223-4680-8c14-f7fb4370c972 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:54.860996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:54.908431 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bf612-6317-4d76-8112-e0ce4ed41435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:54.909065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bf612-6317-4d76-8112-e0ce4ed41435 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:54.909340 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bf612-6317-4d76-8112-e0ce4ed41435 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:54.909612 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bf612-6317-4d76-8112-e0ce4ed41435 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:54.960507 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d548cb8-c56c-4933-ba44-c86b0284bd3e None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:54.984779 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bf612-6317-4d76-8112-e0ce4ed41435 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:54.986269 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bf612-6317-4d76-8112-e0ce4ed41435 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:55.048849 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddffe4-e640-4039-9188-1678a1211849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:55.050155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddffe4-e640-4039-9188-1678a1211849 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:55.050795 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddffe4-e640-4039-9188-1678a1211849 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:55.051296 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddffe4-e640-4039-9188-1678a1211849 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:55.052634 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddffe4-e640-4039-9188-1678a1211849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:55.053335 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddffe4-e640-4039-9188-1678a1211849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:55.056311 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddffe4-e640-4039-9188-1678a1211849 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:55.077195 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:55.104114 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 319/629] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:35:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:55.115759 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddffe4-e640-4039-9188-1678a1211849 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:55.117521 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 311/630] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 12:35:55.129961 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bf612-6317-4d76-8112-e0ce4ed41435 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:55.243893 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bf612-6317-4d76-8112-e0ce4ed41435 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:55.283744 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5adf1-a8a6-441b-a493-6aa674b6e11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:55.284442 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5adf1-a8a6-441b-a493-6aa674b6e11a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:55.284720 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5adf1-a8a6-441b-a493-6aa674b6e11a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:55.284988 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5adf1-a8a6-441b-a493-6aa674b6e11a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:55.296550 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c5adf1-a8a6-441b-a493-6aa674b6e11a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:55.298151 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c5adf1-a8a6-441b-a493-6aa674b6e11a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-99c5adf1-a8a6-441b-a493-6aa674b6e11a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:55.315901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:55.317700 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:55.319507 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:55.319507 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:55.319507 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:55.321877 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 312/631] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:55.323523 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5adf1-a8a6-441b-a493-6aa674b6e11a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:55.340519 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddffe4-e640-4039-9188-1678a1211849 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:55.352301 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:55.378030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:55.416920 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5018a1b-5f60-4b71-ad6b-6008f19bab81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:55.418733 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5018a1b-5f60-4b71-ad6b-6008f19bab81 None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:55.419558 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5018a1b-5f60-4b71-ad6b-6008f19bab81 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:55.420338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5018a1b-5f60-4b71-ad6b-6008f19bab81 None admin] PATH_INFO: `/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/332fdff1d02a4664923ddbe0478eaa7a/roles/943fc0edf23046a69ba6102f2c1e544c` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:55.438495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5adf1-a8a6-441b-a493-6aa674b6e11a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:55.452262 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5018a1b-5f60-4b71-ad6b-6008f19bab81 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fb003a695490faa0077bbbe5e7d77, user_id=332fdff1d02a4664923ddbe0478eaa7a, role_id=943fc0edf23046a69ba6102f2c1e544c)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:55.455781 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5018a1b-5f60-4b71-ad6b-6008f19bab81 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:55.476837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41dc6f-8426-4b8f-a53c-340caff8d3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:55.477603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41dc6f-8426-4b8f-a53c-340caff8d3ce None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:55.477971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41dc6f-8426-4b8f-a53c-340caff8d3ce None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:55.478150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41dc6f-8426-4b8f-a53c-340caff8d3ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:55.483482 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 320/632] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:35:55 2022] PUT /identity/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/332fdff1d02a4664923ddbe0478eaa7a/roles/943fc0edf23046a69ba6102f2c1e544c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:35:55.485647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5018a1b-5f60-4b71-ad6b-6008f19bab81 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:55.498001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41dc6f-8426-4b8f-a53c-340caff8d3ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-9b41dc6f-8426-4b8f-a53c-340caff8d3ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:55.512085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:55.513963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:55.515580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 313/633] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:55.520140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41dc6f-8426-4b8f-a53c-340caff8d3ce None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:55.623919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:55.644259 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41dc6f-8426-4b8f-a53c-340caff8d3ce None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:55.685251 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8e57e-562a-4244-a5a6-5117e6db751c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:55.685888 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8e57e-562a-4244-a5a6-5117e6db751c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:55.686220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8e57e-562a-4244-a5a6-5117e6db751c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:55.686493 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8e57e-562a-4244-a5a6-5117e6db751c None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:55.687192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8e57e-562a-4244-a5a6-5117e6db751c None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:55.687492 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8e57e-562a-4244-a5a6-5117e6db751c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:55.688903 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8e57e-562a-4244-a5a6-5117e6db751c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:55.699529 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 314/634] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:55.710095 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8e57e-562a-4244-a5a6-5117e6db751c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:55.761725 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5018a1b-5f60-4b71-ad6b-6008f19bab81 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:55.831943 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8e57e-562a-4244-a5a6-5117e6db751c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:55.847412 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3f22c-918f-41d6-9635-33858d819789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:55.849160 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3f22c-918f-41d6-9635-33858d819789 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:55.849980 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3f22c-918f-41d6-9635-33858d819789 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:55.850663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3f22c-918f-41d6-9635-33858d819789 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:55.852134 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3f22c-918f-41d6-9635-33858d819789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:55.852895 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3f22c-918f-41d6-9635-33858d819789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:55.856210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3f22c-918f-41d6-9635-33858d819789 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:55.881984 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 321/635] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:55.892944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3f22c-918f-41d6-9635-33858d819789 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:55.897054 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5ea71-2250-4bc5-835e-ffcdf9d3526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:55.898255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5ea71-2250-4bc5-835e-ffcdf9d3526c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:55.898704 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5ea71-2250-4bc5-835e-ffcdf9d3526c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:55.899166 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5ea71-2250-4bc5-835e-ffcdf9d3526c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:55.915443 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5ea71-2250-4bc5-835e-ffcdf9d3526c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:55.917880 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5ea71-2250-4bc5-835e-ffcdf9d3526c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:55.930975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-d6a5ea71-2250-4bc5-835e-ffcdf9d3526c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:55.952024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:55.953106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:55.954087 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:55.954087 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:55.954087 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:55.954087 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 315/636] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:55.961859 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5ea71-2250-4bc5-835e-ffcdf9d3526c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:56.000835 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:56.152365 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3f22c-918f-41d6-9635-33858d819789 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:56.181227 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5ea71-2250-4bc5-835e-ffcdf9d3526c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:56.218924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be57202-537a-4c80-910d-48c6cbf5724e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:56.219633 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be57202-537a-4c80-910d-48c6cbf5724e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:56.219910 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be57202-537a-4c80-910d-48c6cbf5724e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:56.220180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be57202-537a-4c80-910d-48c6cbf5724e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:56.236561 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be57202-537a-4c80-910d-48c6cbf5724e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:56.239738 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be57202-537a-4c80-910d-48c6cbf5724e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:56.250540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dfd78-f3d8-45fc-8220-e12d03605e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:56.251722 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dfd78-f3d8-45fc-8220-e12d03605e0e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:56.252292 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dfd78-f3d8-45fc-8220-e12d03605e0e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:56.252864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dfd78-f3d8-45fc-8220-e12d03605e0e None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:56.254214 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dfd78-f3d8-45fc-8220-e12d03605e0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:56.254793 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dfd78-f3d8-45fc-8220-e12d03605e0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:56.257457 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dfd78-f3d8-45fc-8220-e12d03605e0e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-7be57202-537a-4c80-910d-48c6cbf5724e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:35:56.264998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:56.265935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:56.266841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:35:56.266841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:35:56.266841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:35:56.266841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:35:56.266841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:35:56.266841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:35:56.266841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:56.266841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:56.274227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 322/637] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:56.276326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e579034-ec34-4ffd-97d3-5fd6a6075a10 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:56.276633 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e579034-ec34-4ffd-97d3-5fd6a6075a10 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:56.276899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e579034-ec34-4ffd-97d3-5fd6a6075a10 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:56.283947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 323/638] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:56.285496 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e579034-ec34-4ffd-97d3-5fd6a6075a10 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:56.286711 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 316/639] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:56.298093 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c34f5-1528-4591-a7c6-225a79da4e04 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:56.298093 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c34f5-1528-4591-a7c6-225a79da4e04 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:56.298093 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c34f5-1528-4591-a7c6-225a79da4e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:56.328489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-907c34f5-1528-4591-a7c6-225a79da4e04 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:56.331008 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-907c34f5-1528-4591-a7c6-225a79da4e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:56.408226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:56.410752 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e579034-ec34-4ffd-97d3-5fd6a6075a10 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:56.440478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:56.455542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db936c-470f-42eb-843d-355e61210145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:56.456617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db936c-470f-42eb-843d-355e61210145 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:56.457107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db936c-470f-42eb-843d-355e61210145 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:56.457686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db936c-470f-42eb-843d-355e61210145 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:56.458823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db936c-470f-42eb-843d-355e61210145 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:56.459349 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db936c-470f-42eb-843d-355e61210145 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:56.461109 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db936c-470f-42eb-843d-355e61210145 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:56.467432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 317/640] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:35:56.469014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c34f5-1528-4591-a7c6-225a79da4e04 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:56.476934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 324/641] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:56.494443 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db936c-470f-42eb-843d-355e61210145 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:56.632436 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db936c-470f-42eb-843d-355e61210145 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:56.679766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88bd15-e3d6-4739-b7df-bc15ab6ae182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:56.680788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88bd15-e3d6-4739-b7df-bc15ab6ae182 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:56.682060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88bd15-e3d6-4739-b7df-bc15ab6ae182 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:56.682586 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88bd15-e3d6-4739-b7df-bc15ab6ae182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:56.683561 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88bd15-e3d6-4739-b7df-bc15ab6ae182 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=f2f4f9541f89433cad52e826794b049f) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:56.684412 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88bd15-e3d6-4739-b7df-bc15ab6ae182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:56.686254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88bd15-e3d6-4739-b7df-bc15ab6ae182 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:56.695271 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 325/642] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:35:56 2022] GET /identity/v3/role_assignments?user.id=f2f4f9541f89433cad52e826794b049f&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:56.785144 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c34f5-1528-4591-a7c6-225a79da4e04 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:56.865947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127cc57-964b-4b07-8ecf-7e812d1ec94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:56.867206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127cc57-964b-4b07-8ecf-7e812d1ec94e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:56.867716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127cc57-964b-4b07-8ecf-7e812d1ec94e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:56.868220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127cc57-964b-4b07-8ecf-7e812d1ec94e None admin] PATH_INFO: `/v3/users/nova` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:56.884050 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b127cc57-964b-4b07-8ecf-7e812d1ec94e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:56.887421 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b127cc57-964b-4b07-8ecf-7e812d1ec94e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-b127cc57-964b-4b07-8ecf-7e812d1ec94e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:56.911447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:56.912604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:56.913591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:56.913591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:56.913591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:56.913591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:56.913591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:56.913591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:56.913591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:56.913591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:56.913591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:35:56.913591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:56.914192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 318/643] 10.222.0.33 () {56 vars in 1093 bytes} [Tue Oct 4 12:35:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:56.920940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88bd15-e3d6-4739-b7df-bc15ab6ae182 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:56.955913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:57.058604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127cc57-964b-4b07-8ecf-7e812d1ec94e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:57.151106 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88bd15-e3d6-4739-b7df-bc15ab6ae182 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:57.184714 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:57.230226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732fc8c-b492-4cdc-8f99-2ce60de391f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:57.231698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732fc8c-b492-4cdc-8f99-2ce60de391f9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:57.231698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732fc8c-b492-4cdc-8f99-2ce60de391f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:57.232040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732fc8c-b492-4cdc-8f99-2ce60de391f9 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:57.233804 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732fc8c-b492-4cdc-8f99-2ce60de391f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:57.233930 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732fc8c-b492-4cdc-8f99-2ce60de391f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:57.236525 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3732fc8c-b492-4cdc-8f99-2ce60de391f9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:57.277500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127cc57-964b-4b07-8ecf-7e812d1ec94e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:57.292908 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 326/644] 10.222.0.33 () {56 vars in 1133 bytes} [Tue Oct 4 12:35:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:35:57.295214 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654fb19-fef9-4986-a1e2-f2f3fba9b27c None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:57.295781 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654fb19-fef9-4986-a1e2-f2f3fba9b27c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:57.296326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654fb19-fef9-4986-a1e2-f2f3fba9b27c None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:57.299698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 327/645] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:57.302305 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654fb19-fef9-4986-a1e2-f2f3fba9b27c None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:57.327842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20467fb8-2d86-43ec-8f8f-c99af3637879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:57.328567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20467fb8-2d86-43ec-8f8f-c99af3637879 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:57.328844 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20467fb8-2d86-43ec-8f8f-c99af3637879 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:57.329115 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20467fb8-2d86-43ec-8f8f-c99af3637879 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:57.344289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20467fb8-2d86-43ec-8f8f-c99af3637879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-20467fb8-2d86-43ec-8f8f-c99af3637879 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:57.350592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:57.353882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 319/646] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:35:57.356304 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a226d-3d9d-4e5b-a830-3af03cf33ee3 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:57.357909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a226d-3d9d-4e5b-a830-3af03cf33ee3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:57.357909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a226d-3d9d-4e5b-a830-3af03cf33ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:57.381958 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b44a226d-3d9d-4e5b-a830-3af03cf33ee3 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:57.383346 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b44a226d-3d9d-4e5b-a830-3af03cf33ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:57.473818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:57.477137 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 320/647] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:57.479028 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a226d-3d9d-4e5b-a830-3af03cf33ee3 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:57.545248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:57.581861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654fb19-fef9-4986-a1e2-f2f3fba9b27c None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:57.682619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddad93-d277-4865-b911-94a3f56d7688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:57.684080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddad93-d277-4865-b911-94a3f56d7688 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:57.684578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddad93-d277-4865-b911-94a3f56d7688 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:57.685045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eddad93-d277-4865-b911-94a3f56d7688 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:57.740048 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a226d-3d9d-4e5b-a830-3af03cf33ee3 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:57.828216 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94439111-731e-4a75-9864-cf6aecaac8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:57.828986 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94439111-731e-4a75-9864-cf6aecaac8fd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:57.829158 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94439111-731e-4a75-9864-cf6aecaac8fd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:57.829395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94439111-731e-4a75-9864-cf6aecaac8fd None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:57.830117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94439111-731e-4a75-9864-cf6aecaac8fd None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:57.830424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94439111-731e-4a75-9864-cf6aecaac8fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:57.831970 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94439111-731e-4a75-9864-cf6aecaac8fd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:57.843999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eddad93-d277-4865-b911-94a3f56d7688 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:57.846901 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eddad93-d277-4865-b911-94a3f56d7688 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:57.849549 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 321/648] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:57.878095 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94439111-731e-4a75-9864-cf6aecaac8fd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:58.018507 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:58.045419 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 328/649] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:35:58.053996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddad93-d277-4865-b911-94a3f56d7688 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:58.147764 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:58.153562 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94439111-731e-4a75-9864-cf6aecaac8fd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:58.195555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e430b95-7af0-4838-a2e6-fe5da1b0483c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:58.196454 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e430b95-7af0-4838-a2e6-fe5da1b0483c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:58.196875 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e430b95-7af0-4838-a2e6-fe5da1b0483c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:58.197854 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e430b95-7af0-4838-a2e6-fe5da1b0483c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:58.216263 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e430b95-7af0-4838-a2e6-fe5da1b0483c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:58.219025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e430b95-7af0-4838-a2e6-fe5da1b0483c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0e430b95-7af0-4838-a2e6-fe5da1b0483c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:58.242088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:58.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:58.244322 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:58.244322 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:58.244322 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:58.244753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 322/650] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:58.246274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e430b95-7af0-4838-a2e6-fe5da1b0483c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:58.251486 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eddad93-d277-4865-b911-94a3f56d7688 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:58.297556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ded27-717a-49a6-88bf-b2e1a25a81bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:58.298551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ded27-717a-49a6-88bf-b2e1a25a81bd None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:58.299015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ded27-717a-49a6-88bf-b2e1a25a81bd None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:58.299437 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ded27-717a-49a6-88bf-b2e1a25a81bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:58.312885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ded27-717a-49a6-88bf-b2e1a25a81bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:58.314944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ded27-717a-49a6-88bf-b2e1a25a81bd None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-a83ded27-717a-49a6-88bf-b2e1a25a81bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:58.334093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:58.335911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:58.337461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:58.337461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:58.337461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:58.340666 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 329/651] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:58.341896 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ded27-717a-49a6-88bf-b2e1a25a81bd None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:58.371487 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:58.475040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ded27-717a-49a6-88bf-b2e1a25a81bd None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:58.516839 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e430b95-7af0-4838-a2e6-fe5da1b0483c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:58.521544 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfcab1-ebfe-4f2a-b7f7-450df480daa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:58.522233 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfcab1-ebfe-4f2a-b7f7-450df480daa2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:58.522511 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfcab1-ebfe-4f2a-b7f7-450df480daa2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:58.522832 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfcab1-ebfe-4f2a-b7f7-450df480daa2 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:58.523507 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfcab1-ebfe-4f2a-b7f7-450df480daa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:58.523814 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfcab1-ebfe-4f2a-b7f7-450df480daa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:58.525722 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfcab1-ebfe-4f2a-b7f7-450df480daa2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:58.549609 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 330/652] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:58.551629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfcab1-ebfe-4f2a-b7f7-450df480daa2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:58.573782 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703d631-f2d3-428f-bb1b-536e3909dc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:58.574503 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703d631-f2d3-428f-bb1b-536e3909dc28 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:58.574776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703d631-f2d3-428f-bb1b-536e3909dc28 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:58.575047 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703d631-f2d3-428f-bb1b-536e3909dc28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:58.590676 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703d631-f2d3-428f-bb1b-536e3909dc28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c703d631-f2d3-428f-bb1b-536e3909dc28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:35:58.597173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:58.598939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:58.600518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:58.600518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:58.600518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:58.600518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:58.600518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:35:58.600518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:35:58.600518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:35:58.600518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:35:58.600518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:35:58.600518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:35:58.600518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:35:58.600518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:58.602974 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 323/653] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:35:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:58.604067 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703d631-f2d3-428f-bb1b-536e3909dc28 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:58.624576 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:58.712304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:58.742755 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfcab1-ebfe-4f2a-b7f7-450df480daa2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:58.777411 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703d631-f2d3-428f-bb1b-536e3909dc28 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:58.799223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea7f98-ce78-47de-bcb8-7c0eeac25a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:58.800179 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea7f98-ce78-47de-bcb8-7c0eeac25a45 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:58.800634 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea7f98-ce78-47de-bcb8-7c0eeac25a45 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:58.801071 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea7f98-ce78-47de-bcb8-7c0eeac25a45 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:58.802151 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea7f98-ce78-47de-bcb8-7c0eeac25a45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:58.802650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea7f98-ce78-47de-bcb8-7c0eeac25a45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:58.804627 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea7f98-ce78-47de-bcb8-7c0eeac25a45 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:58.830149 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 331/654] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:58.830149 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea7f98-ce78-47de-bcb8-7c0eeac25a45 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:58.845992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd70b5-50e5-4026-9eea-274940a2f1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:58.846830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd70b5-50e5-4026-9eea-274940a2f1ac None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:58.846969 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd70b5-50e5-4026-9eea-274940a2f1ac None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:58.847341 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd70b5-50e5-4026-9eea-274940a2f1ac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:58.856532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd70b5-50e5-4026-9eea-274940a2f1ac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:58.858002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd70b5-50e5-4026-9eea-274940a2f1ac None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-aadd70b5-50e5-4026-9eea-274940a2f1ac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:35:58.872429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:58.874462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 324/655] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:35:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe19634-4ab0-46fa-9b27-4348fb39a5ce None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:58.876378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe19634-4ab0-46fa-9b27-4348fb39a5ce None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:58.877775 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe19634-4ab0-46fa-9b27-4348fb39a5ce None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:58.877775 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 325/656] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:35:58.878438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe19634-4ab0-46fa-9b27-4348fb39a5ce None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:59.005697 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:59.008383 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea7f98-ce78-47de-bcb8-7c0eeac25a45 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:59.063489 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c6fd5-91e0-4a79-a22b-e35a7664931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:59.063962 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe19634-4ab0-46fa-9b27-4348fb39a5ce None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:59.064148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c6fd5-91e0-4a79-a22b-e35a7664931a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:59.064406 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c6fd5-91e0-4a79-a22b-e35a7664931a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:59.064660 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c6fd5-91e0-4a79-a22b-e35a7664931a None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:59.065286 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c6fd5-91e0-4a79-a22b-e35a7664931a None admin] RBAC: Adding query filter params (name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:59.065580 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c6fd5-91e0-4a79-a22b-e35a7664931a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:59.067234 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c6fd5-91e0-4a79-a22b-e35a7664931a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:59.086015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 332/657] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:35:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:59.086015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c6fd5-91e0-4a79-a22b-e35a7664931a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:59.090412 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:59.125539 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0a5b2-2cd5-4959-a4db-0c8187496f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:59.126622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0a5b2-2cd5-4959-a4db-0c8187496f43 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:59.127111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0a5b2-2cd5-4959-a4db-0c8187496f43 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:59.127590 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0a5b2-2cd5-4959-a4db-0c8187496f43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:59.141151 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0a5b2-2cd5-4959-a4db-0c8187496f43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:59.143985 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0a5b2-2cd5-4959-a4db-0c8187496f43 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-10f0a5b2-2cd5-4959-a4db-0c8187496f43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:35:59.166613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:59.167567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:59.168438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:35:59.168438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:35:59.168438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:35:59.168438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:35:59.168438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:35:59.168438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:35:59.168438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:35:59.168438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:35:59.168849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 326/658] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:35:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:59.170639 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa64ea1-82b9-45c2-a807-e255a390b644 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:59.171118 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa64ea1-82b9-45c2-a807-e255a390b644 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:59.171582 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa64ea1-82b9-45c2-a807-e255a390b644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:59.202626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4fa64ea1-82b9-45c2-a807-e255a390b644 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:35:59.204616 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4fa64ea1-82b9-45c2-a807-e255a390b644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:35:59.251659 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c6fd5-91e0-4a79-a22b-e35a7664931a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:59.296945 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04d75e-780f-4dfb-8e81-5465ec69968d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:59.297577 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04d75e-780f-4dfb-8e81-5465ec69968d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:59.297875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04d75e-780f-4dfb-8e81-5465ec69968d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:59.298156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04d75e-780f-4dfb-8e81-5465ec69968d None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:59.298913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04d75e-780f-4dfb-8e81-5465ec69968d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:59.299217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04d75e-780f-4dfb-8e81-5465ec69968d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:59.300730 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04d75e-780f-4dfb-8e81-5465ec69968d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:59.336336 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 333/659] 10.222.0.33 () {56 vars in 1137 bytes} [Tue Oct 4 12:35:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:59.336336 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04d75e-780f-4dfb-8e81-5465ec69968d None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:59.337559 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 327/660] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:35:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:35:59.339950 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa64ea1-82b9-45c2-a807-e255a390b644 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:59.461998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:59.479222 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:59.559107 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa64ea1-82b9-45c2-a807-e255a390b644 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:59.562232 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04d75e-780f-4dfb-8e81-5465ec69968d None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:59.626009 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8b6ae-631d-4bf9-a8e4-a18fd40b7425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:59.627115 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8b6ae-631d-4bf9-a8e4-a18fd40b7425 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:59.627580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8b6ae-631d-4bf9-a8e4-a18fd40b7425 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:59.628038 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8b6ae-631d-4bf9-a8e4-a18fd40b7425 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:59.629210 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8b6ae-631d-4bf9-a8e4-a18fd40b7425 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:59.629726 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8b6ae-631d-4bf9-a8e4-a18fd40b7425 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:59.632263 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8b6ae-631d-4bf9-a8e4-a18fd40b7425 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:59.637930 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bb69e-0688-4048-859d-5de2a81db58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:59.639355 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bb69e-0688-4048-859d-5de2a81db58d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:59.640058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bb69e-0688-4048-859d-5de2a81db58d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:59.640747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bb69e-0688-4048-859d-5de2a81db58d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:59.660987 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 328/661] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:35:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:59.660987 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8b6ae-631d-4bf9-a8e4-a18fd40b7425 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:59.671821 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bb69e-0688-4048-859d-5de2a81db58d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:59.676245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bb69e-0688-4048-859d-5de2a81db58d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-3b0bb69e-0688-4048-859d-5de2a81db58d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:35:59.708254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:35:59.710133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:35:59.712616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:35:59.712616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:35:59.712616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:35:59.721689 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 334/662] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:35:59.723770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bb69e-0688-4048-859d-5de2a81db58d None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:35:59.848903 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8b6ae-631d-4bf9-a8e4-a18fd40b7425 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:59.887209 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:59.903661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90ef73-f6d9-47d5-af2b-b1dfae8a956d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:59.904468 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90ef73-f6d9-47d5-af2b-b1dfae8a956d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:59.904853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90ef73-f6d9-47d5-af2b-b1dfae8a956d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:59.905182 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90ef73-f6d9-47d5-af2b-b1dfae8a956d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:59.906505 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bb69e-0688-4048-859d-5de2a81db58d None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:35:59.922499 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:35:59.952402 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572add5-18b3-42fb-995b-70c6da9320b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:35:59.953176 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572add5-18b3-42fb-995b-70c6da9320b4 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:35:59.953621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572add5-18b3-42fb-995b-70c6da9320b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:35:59.954086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1572add5-18b3-42fb-995b-70c6da9320b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:35:59.955335 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572add5-18b3-42fb-995b-70c6da9320b4 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=02260480264b4f8f8a7e5cb647ac5d9e) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:35:59.955837 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572add5-18b3-42fb-995b-70c6da9320b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:35:59.958357 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1572add5-18b3-42fb-995b-70c6da9320b4 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:35:59.968751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 335/663] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:35:59 2022] GET /identity/v3/role_assignments?user.id=02260480264b4f8f8a7e5cb647ac5d9e&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:35:59.969947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572add5-18b3-42fb-995b-70c6da9320b4 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:00.026322 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90ef73-f6d9-47d5-af2b-b1dfae8a956d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:00.030553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90ef73-f6d9-47d5-af2b-b1dfae8a956d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:00.150321 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1572add5-18b3-42fb-995b-70c6da9320b4 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:00.201903 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2da7b1-c302-450c-b186-2f7e990e6573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:00.202843 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2da7b1-c302-450c-b186-2f7e990e6573 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:00.203128 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2da7b1-c302-450c-b186-2f7e990e6573 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:00.203398 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2da7b1-c302-450c-b186-2f7e990e6573 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:00.204221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2da7b1-c302-450c-b186-2f7e990e6573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:00.204874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2da7b1-c302-450c-b186-2f7e990e6573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:00.206600 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2da7b1-c302-450c-b186-2f7e990e6573 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:00.213636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 329/664] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:35:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:36:00.215035 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90ef73-f6d9-47d5-af2b-b1dfae8a956d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:00.222419 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 336/665] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:00.223852 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579084b-f31e-4f01-83da-bd97fff0e2d2 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:00.224138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579084b-f31e-4f01-83da-bd97fff0e2d2 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:00.224930 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579084b-f31e-4f01-83da-bd97fff0e2d2 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:00.225984 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 337/666] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:00.227062 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579084b-f31e-4f01-83da-bd97fff0e2d2 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:00.264219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:00.289235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:00.380954 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90ef73-f6d9-47d5-af2b-b1dfae8a956d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:00.398718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579084b-f31e-4f01-83da-bd97fff0e2d2 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:00.442828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b26d64-5436-4f0e-b843-b51d7e8efe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:00.443909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b26d64-5436-4f0e-b843-b51d7e8efe4c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:00.444395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b26d64-5436-4f0e-b843-b51d7e8efe4c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:00.444896 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b26d64-5436-4f0e-b843-b51d7e8efe4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:00.456268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7761a5-5b05-4198-a246-cedf2ea53822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:00.457120 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7761a5-5b05-4198-a246-cedf2ea53822 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:00.457532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7761a5-5b05-4198-a246-cedf2ea53822 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:00.457946 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7761a5-5b05-4198-a246-cedf2ea53822 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:00.465720 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b26d64-5436-4f0e-b843-b51d7e8efe4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:00.471581 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7761a5-5b05-4198-a246-cedf2ea53822 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:00.473290 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7761a5-5b05-4198-a246-cedf2ea53822 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c4b26d64-5436-4f0e-b843-b51d7e8efe4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:00.475437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:00.476432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:00.477306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:00.477306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:00.477306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:00.477306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:00.477306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:00.477306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:00.477306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:00.477306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:00.477306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:00.477306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:00.477306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:36:00.477306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:00.479024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 330/667] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:36:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:00.479707 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b26d64-5436-4f0e-b843-b51d7e8efe4c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-1f7761a5-5b05-4198-a246-cedf2ea53822 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:00.506893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:00.508133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:00.509328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:00.509328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:00.509328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:00.509328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 338/668] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:00.510992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39917c6-06aa-45e1-8e31-2efcc1ad8d44 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:00.511358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39917c6-06aa-45e1-8e31-2efcc1ad8d44 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:00.511830 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39917c6-06aa-45e1-8e31-2efcc1ad8d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:00.545093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-c39917c6-06aa-45e1-8e31-2efcc1ad8d44 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:00.547389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-c39917c6-06aa-45e1-8e31-2efcc1ad8d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:00.606283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:00.631475 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b26d64-5436-4f0e-b843-b51d7e8efe4c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:00.665323 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 339/669] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:36:00.667734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39917c6-06aa-45e1-8e31-2efcc1ad8d44 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:00.692704 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17ef2e-fe9d-428b-8d60-0a98e9f80311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:00.694323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17ef2e-fe9d-428b-8d60-0a98e9f80311 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:00.695026 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17ef2e-fe9d-428b-8d60-0a98e9f80311 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:00.695716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17ef2e-fe9d-428b-8d60-0a98e9f80311 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:00.701213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:00.711017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17ef2e-fe9d-428b-8d60-0a98e9f80311 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:00.714933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17ef2e-fe9d-428b-8d60-0a98e9f80311 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-8a17ef2e-fe9d-428b-8d60-0a98e9f80311 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:00.738737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:00.739759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:00.740728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:00.740728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:00.740728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:00.740728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:00.740728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:00.740728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:00.740728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:00.740728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:00.740728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 12:36:00.740728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:00.741271 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 331/670] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:36:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:00.742719 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17ef2e-fe9d-428b-8d60-0a98e9f80311 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:00.819548 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39917c6-06aa-45e1-8e31-2efcc1ad8d44 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:00.872109 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88173185-c371-465f-857d-41da34bdb579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:00.873108 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88173185-c371-465f-857d-41da34bdb579 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:00.874295 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88173185-c371-465f-857d-41da34bdb579 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:00.874810 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88173185-c371-465f-857d-41da34bdb579 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:00.875764 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88173185-c371-465f-857d-41da34bdb579 None admin] RBAC: Adding query filter params (name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:00.876572 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88173185-c371-465f-857d-41da34bdb579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:00.878235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88173185-c371-465f-857d-41da34bdb579 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:00.889973 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 340/671] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:36:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:00.891221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88173185-c371-465f-857d-41da34bdb579 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:00.949783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17ef2e-fe9d-428b-8d60-0a98e9f80311 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:01.001230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:01.017922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3168280-01f3-4087-a409-e1b14b51b7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:01.018980 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3168280-01f3-4087-a409-e1b14b51b7b9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:01.019823 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3168280-01f3-4087-a409-e1b14b51b7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:01.020206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3168280-01f3-4087-a409-e1b14b51b7b9 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:01.021427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3168280-01f3-4087-a409-e1b14b51b7b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:01.021963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3168280-01f3-4087-a409-e1b14b51b7b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:01.024593 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3168280-01f3-4087-a409-e1b14b51b7b9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:01.028819 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88173185-c371-465f-857d-41da34bdb579 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:01.051853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 332/672] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:01.053464 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3168280-01f3-4087-a409-e1b14b51b7b9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:01.058596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:01.071575 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2796be-d109-42d3-b686-6c16dab051f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:01.073367 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2796be-d109-42d3-b686-6c16dab051f6 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:01.074067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2796be-d109-42d3-b686-6c16dab051f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:01.074557 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2796be-d109-42d3-b686-6c16dab051f6 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:01.075617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2796be-d109-42d3-b686-6c16dab051f6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:01.076147 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2796be-d109-42d3-b686-6c16dab051f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:01.077874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2796be-d109-42d3-b686-6c16dab051f6 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:01.100389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 341/673] 10.222.0.33 () {56 vars in 1139 bytes} [Tue Oct 4 12:36:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:01.101672 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2796be-d109-42d3-b686-6c16dab051f6 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:01.214926 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2796be-d109-42d3-b686-6c16dab051f6 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:01.258724 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3168280-01f3-4087-a409-e1b14b51b7b9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:01.268069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc25cc7-477b-4ce0-b3a0-c655c44c81c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:01.268769 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc25cc7-477b-4ce0-b3a0-c655c44c81c5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:01.269040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc25cc7-477b-4ce0-b3a0-c655c44c81c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:01.269578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc25cc7-477b-4ce0-b3a0-c655c44c81c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:01.277814 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc25cc7-477b-4ce0-b3a0-c655c44c81c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:01.279342 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc25cc7-477b-4ce0-b3a0-c655c44c81c5 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:01.281598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-cbc25cc7-477b-4ce0-b3a0-c655c44c81c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:01.293737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:01.295595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:01.297226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:01.297226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:01.297226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:01.297226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:01.297226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:01.297226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:01.297226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:01.297226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:01.297226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 342/674] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:01.297226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc25cc7-477b-4ce0-b3a0-c655c44c81c5 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:01.334706 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2be5e-4fad-4770-b10c-0ec6bc5dcdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:01.335732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2be5e-4fad-4770-b10c-0ec6bc5dcdd2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:01.336079 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2be5e-4fad-4770-b10c-0ec6bc5dcdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:01.336352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2be5e-4fad-4770-b10c-0ec6bc5dcdd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:01.348850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2be5e-4fad-4770-b10c-0ec6bc5dcdd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:01.350862 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2be5e-4fad-4770-b10c-0ec6bc5dcdd2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-38e2be5e-4fad-4770-b10c-0ec6bc5dcdd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:01.377947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:01.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:01.383366 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:01.383366 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:01.383366 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:01.383366 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 333/675] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:01.383366 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2be5e-4fad-4770-b10c-0ec6bc5dcdd2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:01.428867 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc25cc7-477b-4ce0-b3a0-c655c44c81c5 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:01.467619 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:01.486709 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6c3c2-a9aa-4c14-903c-69de1092243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:01.487336 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6c3c2-a9aa-4c14-903c-69de1092243e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:01.487739 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6c3c2-a9aa-4c14-903c-69de1092243e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:01.487856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6c3c2-a9aa-4c14-903c-69de1092243e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:01.551321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2be5e-4fad-4770-b10c-0ec6bc5dcdd2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:01.605333 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:01.611399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6c3c2-a9aa-4c14-903c-69de1092243e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:01.614351 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6c3c2-a9aa-4c14-903c-69de1092243e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:01.618736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ea7c2-7b14-4204-9ff2-ec8eb376de7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:01.621619 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ea7c2-7b14-4204-9ff2-ec8eb376de7e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:01.622080 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ea7c2-7b14-4204-9ff2-ec8eb376de7e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:01.622487 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ea7c2-7b14-4204-9ff2-ec8eb376de7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:01.643104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ea7c2-7b14-4204-9ff2-ec8eb376de7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:01.644899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ea7c2-7b14-4204-9ff2-ec8eb376de7e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-249ea7c2-7b14-4204-9ff2-ec8eb376de7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:01.664757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:01.666686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:01.668512 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:01.668512 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:01.668512 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:01.668512 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 334/676] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:01.668512 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ea7c2-7b14-4204-9ff2-ec8eb376de7e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:01.747413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 343/677] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:36:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:36:01.752564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6c3c2-a9aa-4c14-903c-69de1092243e None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:01.842198 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ea7c2-7b14-4204-9ff2-ec8eb376de7e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:01.856516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:01.893142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:01.899420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6c3c2-a9aa-4c14-903c-69de1092243e None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:01.911430 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b925ea-6576-48f5-87a6-03f2cd821cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:01.912402 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b925ea-6576-48f5-87a6-03f2cd821cda None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:01.912853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b925ea-6576-48f5-87a6-03f2cd821cda None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:01.913423 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b925ea-6576-48f5-87a6-03f2cd821cda None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:01.914301 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b925ea-6576-48f5-87a6-03f2cd821cda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:01.914685 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b925ea-6576-48f5-87a6-03f2cd821cda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:01.917276 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b925ea-6576-48f5-87a6-03f2cd821cda None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:01.942484 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 335/678] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:01.943832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b925ea-6576-48f5-87a6-03f2cd821cda None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:01.966978 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed736f-4497-4dbd-b035-557501448d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:01.967835 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed736f-4497-4dbd-b035-557501448d3e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:01.968246 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed736f-4497-4dbd-b035-557501448d3e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:01.968651 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed736f-4497-4dbd-b035-557501448d3e None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:01.969479 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed736f-4497-4dbd-b035-557501448d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:01.969924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed736f-4497-4dbd-b035-557501448d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:01.971949 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed736f-4497-4dbd-b035-557501448d3e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:02.000702 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 344/679] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:02.002224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed736f-4497-4dbd-b035-557501448d3e None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:02.127235 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b925ea-6576-48f5-87a6-03f2cd821cda None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:02.170461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed736f-4497-4dbd-b035-557501448d3e None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:02.197899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a5d4b-20ca-499c-aba1-bf519101f434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:02.198939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a5d4b-20ca-499c-aba1-bf519101f434 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:02.199398 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a5d4b-20ca-499c-aba1-bf519101f434 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:02.199899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a5d4b-20ca-499c-aba1-bf519101f434 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:02.201342 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a5d4b-20ca-499c-aba1-bf519101f434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:02.202751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a5d4b-20ca-499c-aba1-bf519101f434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:02.206921 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a5d4b-20ca-499c-aba1-bf519101f434 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:02.218312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a119317-ae8c-46ba-a9ab-f732e90023f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:02.219361 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a119317-ae8c-46ba-a9ab-f732e90023f8 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:02.219845 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a119317-ae8c-46ba-a9ab-f732e90023f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:02.220328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a119317-ae8c-46ba-a9ab-f732e90023f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:02.235185 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 336/680] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:02.235185 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a5d4b-20ca-499c-aba1-bf519101f434 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:02.244442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a119317-ae8c-46ba-a9ab-f732e90023f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:02.247382 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a119317-ae8c-46ba-a9ab-f732e90023f8 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:02.273789 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-0a119317-ae8c-46ba-a9ab-f732e90023f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:02.286818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:02.288680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:02.291197 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:02.291197 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:02.291197 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:02.291197 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 345/681] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:02.291197 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a119317-ae8c-46ba-a9ab-f732e90023f8 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:02.317814 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:02.368393 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a5d4b-20ca-499c-aba1-bf519101f434 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:02.412726 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4524f-76a7-4a50-957c-6b4ca5d50ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:02.413753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4524f-76a7-4a50-957c-6b4ca5d50ee7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:02.414227 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4524f-76a7-4a50-957c-6b4ca5d50ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:02.414704 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4524f-76a7-4a50-957c-6b4ca5d50ee7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:02.416311 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4524f-76a7-4a50-957c-6b4ca5d50ee7 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=b131edeaa17e4e10af385339d463a512) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:02.416875 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4524f-76a7-4a50-957c-6b4ca5d50ee7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:02.419735 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4524f-76a7-4a50-957c-6b4ca5d50ee7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:02.434685 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 337/682] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:36:02 2022] GET /identity/v3/role_assignments?user.id=b131edeaa17e4e10af385339d463a512&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:02.434685 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4524f-76a7-4a50-957c-6b4ca5d50ee7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:02.476988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a119317-ae8c-46ba-a9ab-f732e90023f8 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:02.538226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab2e7c-37a6-4435-afd6-5781f2bc2ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:02.539009 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab2e7c-37a6-4435-afd6-5781f2bc2ca9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:02.539564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab2e7c-37a6-4435-afd6-5781f2bc2ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:02.540010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab2e7c-37a6-4435-afd6-5781f2bc2ca9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:02.550714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab2e7c-37a6-4435-afd6-5781f2bc2ca9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:02.552468 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab2e7c-37a6-4435-afd6-5781f2bc2ca9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-13ab2e7c-37a6-4435-afd6-5781f2bc2ca9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:02.567775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:02.568791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:02.569714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:02.569714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:02.569714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:02.569714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:02.569714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:02.569714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:02.569714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:02.569714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:02.569714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 12:36:02.569714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:02.569714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 346/683] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:36:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:02.571045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab2e7c-37a6-4435-afd6-5781f2bc2ca9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:02.607383 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:02.630876 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4524f-76a7-4a50-957c-6b4ca5d50ee7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:02.683003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726a7d4-48f1-4d80-aaf1-5fde26e4b13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:02.683937 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726a7d4-48f1-4d80-aaf1-5fde26e4b13e None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:02.684325 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726a7d4-48f1-4d80-aaf1-5fde26e4b13e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:02.684742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726a7d4-48f1-4d80-aaf1-5fde26e4b13e None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:02.685689 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726a7d4-48f1-4d80-aaf1-5fde26e4b13e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:02.687545 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726a7d4-48f1-4d80-aaf1-5fde26e4b13e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:02.705919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:02.717673 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 338/684] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:36:02 2022] POST /identity/v3/users => generated 254 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:36:02.720221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ef426-ed68-4334-bb45-e59204e27d51 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:02.720735 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ef426-ed68-4334-bb45-e59204e27d51 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:02.721208 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ef426-ed68-4334-bb45-e59204e27d51 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:02.723808 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 339/685] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:02.725529 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ef426-ed68-4334-bb45-e59204e27d51 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:02.778608 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab2e7c-37a6-4435-afd6-5781f2bc2ca9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:02.844148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde941a4-d270-4600-a819-dbc1bce4363e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:02.845616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde941a4-d270-4600-a819-dbc1bce4363e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:02.846419 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde941a4-d270-4600-a819-dbc1bce4363e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:02.847278 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde941a4-d270-4600-a819-dbc1bce4363e None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:02.848661 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde941a4-d270-4600-a819-dbc1bce4363e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:02.849472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde941a4-d270-4600-a819-dbc1bce4363e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:02.852364 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde941a4-d270-4600-a819-dbc1bce4363e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:02.877203 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 347/686] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:02.879869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20842cc8-0369-4b5e-b410-68109e76d46c None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:02.880640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20842cc8-0369-4b5e-b410-68109e76d46c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:02.881379 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20842cc8-0369-4b5e-b410-68109e76d46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:02.911898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-20842cc8-0369-4b5e-b410-68109e76d46c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:02.914427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-20842cc8-0369-4b5e-b410-68109e76d46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:02.946263 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ef426-ed68-4334-bb45-e59204e27d51 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:03.015460 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bc141-d03f-4162-a036-64a528ad4c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:03.017331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bc141-d03f-4162-a036-64a528ad4c8d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:03.017331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bc141-d03f-4162-a036-64a528ad4c8d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:03.017675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bc141-d03f-4162-a036-64a528ad4c8d None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:03.018926 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bc141-d03f-4162-a036-64a528ad4c8d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:03.019711 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bc141-d03f-4162-a036-64a528ad4c8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:03.023669 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bc141-d03f-4162-a036-64a528ad4c8d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:03.049150 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 348/687] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:36:03.051164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20842cc8-0369-4b5e-b410-68109e76d46c None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:03.054487 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 340/688] 10.222.0.33 () {56 vars in 1135 bytes} [Tue Oct 4 12:36:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:03.069824 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bc141-d03f-4162-a036-64a528ad4c8d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:03.078524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:03.188363 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:03.329918 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20842cc8-0369-4b5e-b410-68109e76d46c None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:03.372520 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bc141-d03f-4162-a036-64a528ad4c8d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:03.442685 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbe508-f189-434c-87fb-a9b3e382213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:03.443570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbe508-f189-434c-87fb-a9b3e382213b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:03.443973 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbe508-f189-434c-87fb-a9b3e382213b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:03.444491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbe508-f189-434c-87fb-a9b3e382213b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:03.456422 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbe508-f189-434c-87fb-a9b3e382213b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:03.458977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbe508-f189-434c-87fb-a9b3e382213b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:03.468410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2ecdb-c66f-4247-9bb5-313b42763a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:03.469862 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2ecdb-c66f-4247-9bb5-313b42763a50 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:03.470614 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2ecdb-c66f-4247-9bb5-313b42763a50 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:03.471242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2ecdb-c66f-4247-9bb5-313b42763a50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-6fbbe508-f189-434c-87fb-a9b3e382213b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:03.486073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:03.487789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:03.489480 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:03.489480 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:03.489480 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:03.489480 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:03.489480 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:03.489480 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:03.489480 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:03.489480 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:03.489480 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 349/689] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:03.492010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbe508-f189-434c-87fb-a9b3e382213b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:03.635432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2ecdb-c66f-4247-9bb5-313b42763a50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:03.638983 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2ecdb-c66f-4247-9bb5-313b42763a50 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:03.641300 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbe508-f189-434c-87fb-a9b3e382213b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:03.646438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:03.669544 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:03.687724 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e9b25-fb18-4765-abda-2034627fcae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:03.688358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e9b25-fb18-4765-abda-2034627fcae0 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:03.688639 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e9b25-fb18-4765-abda-2034627fcae0 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:03.688916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e9b25-fb18-4765-abda-2034627fcae0 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:03.689621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e9b25-fb18-4765-abda-2034627fcae0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:03.689930 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e9b25-fb18-4765-abda-2034627fcae0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:03.691466 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e9b25-fb18-4765-abda-2034627fcae0 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:03.708529 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 350/690] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:03.709312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e9b25-fb18-4765-abda-2034627fcae0 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:03.828380 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e9b25-fb18-4765-abda-2034627fcae0 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:03.846218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 341/691] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:36:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 772 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 12:36:03.847883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2ecdb-c66f-4247-9bb5-313b42763a50 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:03.869522 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede22617-83a7-4706-a68f-2f74387314b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:03.870338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede22617-83a7-4706-a68f-2f74387314b7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:03.870676 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede22617-83a7-4706-a68f-2f74387314b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:03.870950 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede22617-83a7-4706-a68f-2f74387314b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:03.883413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede22617-83a7-4706-a68f-2f74387314b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-ede22617-83a7-4706-a68f-2f74387314b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:03.889253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:03.891190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 351/692] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:36:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:03.892869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede22617-83a7-4706-a68f-2f74387314b7 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:03.941324 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:04.014962 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede22617-83a7-4706-a68f-2f74387314b7 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:04.039487 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2ecdb-c66f-4247-9bb5-313b42763a50 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:04.043891 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:04.059002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187dc889-5b0d-44bf-8cd4-7781c1aae3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:04.059658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187dc889-5b0d-44bf-8cd4-7781c1aae3b1 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:04.059944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187dc889-5b0d-44bf-8cd4-7781c1aae3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:04.060210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187dc889-5b0d-44bf-8cd4-7781c1aae3b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:04.072986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187dc889-5b0d-44bf-8cd4-7781c1aae3b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:04.074617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187dc889-5b0d-44bf-8cd4-7781c1aae3b1 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-187dc889-5b0d-44bf-8cd4-7781c1aae3b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:04.093399 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:04.096892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:04.100344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:04.100344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:04.100344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:04.100344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 352/693] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:04.100344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187dc889-5b0d-44bf-8cd4-7781c1aae3b1 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:04.113767 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc91067-668b-46dc-8282-cc5a439b6511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:04.114877 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc91067-668b-46dc-8282-cc5a439b6511 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:04.115352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc91067-668b-46dc-8282-cc5a439b6511 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:04.115826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc91067-668b-46dc-8282-cc5a439b6511 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:04.117259 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc91067-668b-46dc-8282-cc5a439b6511 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=756aec42473f4bebb15f8d43fe16504d) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:04.117801 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc91067-668b-46dc-8282-cc5a439b6511 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:04.120726 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc91067-668b-46dc-8282-cc5a439b6511 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:04.132858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 342/694] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:36:03 2022] GET /identity/v3/role_assignments?user.id=756aec42473f4bebb15f8d43fe16504d&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:04.135114 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc91067-668b-46dc-8282-cc5a439b6511 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:04.215624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:04.228853 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187dc889-5b0d-44bf-8cd4-7781c1aae3b1 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:04.271479 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0f1da-0b3a-4305-9722-15fafad93a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:04.272490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0f1da-0b3a-4305-9722-15fafad93a5c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:04.272977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0f1da-0b3a-4305-9722-15fafad93a5c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:04.273517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0f1da-0b3a-4305-9722-15fafad93a5c None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:04.274490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0f1da-0b3a-4305-9722-15fafad93a5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:04.275062 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0f1da-0b3a-4305-9722-15fafad93a5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:04.277339 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0f1da-0b3a-4305-9722-15fafad93a5c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:04.288504 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 353/695] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:36:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:04.297620 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0f1da-0b3a-4305-9722-15fafad93a5c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:04.326402 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc91067-668b-46dc-8282-cc5a439b6511 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:04.404553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f37fcb-4cab-45e8-b39c-1de21aa177a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:04.405689 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f37fcb-4cab-45e8-b39c-1de21aa177a4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:04.406352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f37fcb-4cab-45e8-b39c-1de21aa177a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:04.406845 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f37fcb-4cab-45e8-b39c-1de21aa177a4 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:04.408193 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f37fcb-4cab-45e8-b39c-1de21aa177a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:04.408732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f37fcb-4cab-45e8-b39c-1de21aa177a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:04.411612 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f37fcb-4cab-45e8-b39c-1de21aa177a4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:04.466466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 343/696] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:04.479909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f37fcb-4cab-45e8-b39c-1de21aa177a4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:04.542561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:04.601311 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0f1da-0b3a-4305-9722-15fafad93a5c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:04.630177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f37fcb-4cab-45e8-b39c-1de21aa177a4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:04.658886 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f0574-3d7e-4e49-920d-da23272d2e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:04.659553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f0574-3d7e-4e49-920d-da23272d2e44 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:04.659836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f0574-3d7e-4e49-920d-da23272d2e44 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:04.660107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f0574-3d7e-4e49-920d-da23272d2e44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:04.686949 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f0574-3d7e-4e49-920d-da23272d2e44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:04.688441 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f0574-3d7e-4e49-920d-da23272d2e44 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:04.695970 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb965d-677f-4f2a-8332-3353fd6c1742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:04.697270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb965d-677f-4f2a-8332-3353fd6c1742 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:04.697767 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb965d-677f-4f2a-8332-3353fd6c1742 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:04.698282 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb965d-677f-4f2a-8332-3353fd6c1742 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:04.710604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb965d-677f-4f2a-8332-3353fd6c1742 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:04.713822 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb965d-677f-4f2a-8332-3353fd6c1742 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-985f0574-3d7e-4e49-920d-da23272d2e44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:04.719563 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:04.720527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:04.721390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:04.721390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:04.721390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:04.725367 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 354/697] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:04.731100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f0574-3d7e-4e49-920d-da23272d2e44 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-68eb965d-677f-4f2a-8332-3353fd6c1742 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:04.736995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:04.737899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:04.738830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:04.738830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:04.738830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:04.738830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:04.738830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:04.738830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:04.738830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:04.738830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:04.746154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 344/698] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:04.746724 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:04.750850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb965d-677f-4f2a-8332-3353fd6c1742 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:04.964936 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f0574-3d7e-4e49-920d-da23272d2e44 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:04.999372 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:05.005231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccb630-540b-43a0-8807-2e67dc5dc67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:05.006088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccb630-540b-43a0-8807-2e67dc5dc67a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:05.006536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccb630-540b-43a0-8807-2e67dc5dc67a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:05.007449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccb630-540b-43a0-8807-2e67dc5dc67a None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:05.008416 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccb630-540b-43a0-8807-2e67dc5dc67a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:05.008867 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccb630-540b-43a0-8807-2e67dc5dc67a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:05.011151 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ccb630-540b-43a0-8807-2e67dc5dc67a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:05.026533 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 355/699] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:05.032519 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccb630-540b-43a0-8807-2e67dc5dc67a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:05.032785 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb965d-677f-4f2a-8332-3353fd6c1742 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:05.126847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1db67-2112-48e7-aa26-edc81e92db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:05.127448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1db67-2112-48e7-aa26-edc81e92db99 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:05.127726 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1db67-2112-48e7-aa26-edc81e92db99 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:05.127989 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1db67-2112-48e7-aa26-edc81e92db99 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:05.128705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1db67-2112-48e7-aa26-edc81e92db99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:05.129009 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1db67-2112-48e7-aa26-edc81e92db99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:05.130997 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1db67-2112-48e7-aa26-edc81e92db99 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:05.162395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 345/700] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:05.167969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:05.169337 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1db67-2112-48e7-aa26-edc81e92db99 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:05.283618 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccb630-540b-43a0-8807-2e67dc5dc67a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:05.374701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c40440-3fd2-4311-84ef-7673b1740870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:05.375368 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c40440-3fd2-4311-84ef-7673b1740870 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:05.375649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c40440-3fd2-4311-84ef-7673b1740870 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:05.375921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c40440-3fd2-4311-84ef-7673b1740870 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:05.395801 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c40440-3fd2-4311-84ef-7673b1740870 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:05.397340 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c40440-3fd2-4311-84ef-7673b1740870 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:05.419177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1db67-2112-48e7-aa26-edc81e92db99 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-89c40440-3fd2-4311-84ef-7673b1740870 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:05.420031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:05.421098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:05.422126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:05.422126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:05.422126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:05.422126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:05.422126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:05.422126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:05.422126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:05.422126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:05.422126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:36:05.422126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:05.426221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 356/701] 10.222.0.33 () {56 vars in 1093 bytes} [Tue Oct 4 12:36:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:05.428244 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c40440-3fd2-4311-84ef-7673b1740870 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:05.494418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4b21c-e7be-4a3b-b929-cc8e0c5a5571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:05.495421 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4b21c-e7be-4a3b-b929-cc8e0c5a5571 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:05.495878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4b21c-e7be-4a3b-b929-cc8e0c5a5571 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:05.496335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4b21c-e7be-4a3b-b929-cc8e0c5a5571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:05.497584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4b21c-e7be-4a3b-b929-cc8e0c5a5571 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=332fdff1d02a4664923ddbe0478eaa7a) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:05.498139 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4b21c-e7be-4a3b-b929-cc8e0c5a5571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:05.501913 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed4b21c-e7be-4a3b-b929-cc8e0c5a5571 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:05.514605 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 346/702] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:36:05 2022] GET /identity/v3/role_assignments?user.id=332fdff1d02a4664923ddbe0478eaa7a&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 608 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:05.649386 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c40440-3fd2-4311-84ef-7673b1740870 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:05.689158 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:05.700409 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef9bc6c-1dbe-4a65-b023-26a7382e2ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:05.701044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9bc6c-1dbe-4a65-b023-26a7382e2ba2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:05.701323 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9bc6c-1dbe-4a65-b023-26a7382e2ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:05.701599 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef9bc6c-1dbe-4a65-b023-26a7382e2ba2 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:05.702410 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9bc6c-1dbe-4a65-b023-26a7382e2ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:05.702718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9bc6c-1dbe-4a65-b023-26a7382e2ba2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:05.704239 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef9bc6c-1dbe-4a65-b023-26a7382e2ba2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:05.734865 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 357/703] 10.222.0.33 () {56 vars in 1133 bytes} [Tue Oct 4 12:36:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:05.741208 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4b21c-e7be-4a3b-b929-cc8e0c5a5571 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:05.768619 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:05.907179 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4b21c-e7be-4a3b-b929-cc8e0c5a5571 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:05.932899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05755a1-2104-4994-9151-ef54c89dc3b2 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:05.933223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05755a1-2104-4994-9151-ef54c89dc3b2 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:05.933445 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05755a1-2104-4994-9151-ef54c89dc3b2 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:05.935040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 358/704] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:36:05.942937 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db852c-3f15-4ac6-a411-0e0f8a8d156b None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:05.943284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db852c-3f15-4ac6-a411-0e0f8a8d156b None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:05.943486 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db852c-3f15-4ac6-a411-0e0f8a8d156b None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:05.944971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 359/705] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:36:05.947389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b76d49-2f88-4761-9b3d-6d0de26bf099 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:05.947668 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b76d49-2f88-4761-9b3d-6d0de26bf099 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:05.947934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b76d49-2f88-4761-9b3d-6d0de26bf099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:05.953879 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbf839-0c21-432b-aaef-bbb88debd892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:05.955196 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbf839-0c21-432b-aaef-bbb88debd892 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:05.955538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbf839-0c21-432b-aaef-bbb88debd892 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:05.955833 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbf839-0c21-432b-aaef-bbb88debd892 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:05.976219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-86b76d49-2f88-4761-9b3d-6d0de26bf099 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:05.977797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-86b76d49-2f88-4761-9b3d-6d0de26bf099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:06.066562 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbf839-0c21-432b-aaef-bbb88debd892 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:06.068622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbf839-0c21-432b-aaef-bbb88debd892 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:06.088952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:06.096786 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 360/706] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:36:06.101292 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e709209-c056-42bc-9e4f-5b0c31719ec3 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:06.102182 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e709209-c056-42bc-9e4f-5b0c31719ec3 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:06.103872 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e709209-c056-42bc-9e4f-5b0c31719ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:06.134251 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-2e709209-c056-42bc-9e4f-5b0c31719ec3 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:06.135540 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-2e709209-c056-42bc-9e4f-5b0c31719ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:06.248382 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 347/707] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:36:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:36:06.255656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbf839-0c21-432b-aaef-bbb88debd892 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:06.315543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 361/708] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:36:06.452341 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbf839-0c21-432b-aaef-bbb88debd892 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:06.462363 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e709209-c056-42bc-9e4f-5b0c31719ec3 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:06.483996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:06.531575 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:06.551463 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a428c0-d6f0-4f9f-977b-ac32595a0e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:06.552119 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a428c0-d6f0-4f9f-977b-ac32595a0e48 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:06.552396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a428c0-d6f0-4f9f-977b-ac32595a0e48 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:06.552670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a428c0-d6f0-4f9f-977b-ac32595a0e48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:06.577208 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a428c0-d6f0-4f9f-977b-ac32595a0e48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:06.581365 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a428c0-d6f0-4f9f-977b-ac32595a0e48 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-47a428c0-d6f0-4f9f-977b-ac32595a0e48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:06.604134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:06.605359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:06.606426 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:06.606426 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:06.606426 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:06.608227 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 348/709] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:06.613900 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a428c0-d6f0-4f9f-977b-ac32595a0e48 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:06.632317 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e709209-c056-42bc-9e4f-5b0c31719ec3 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:06.687172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf5ebd-7d81-4465-9e25-6ca1ae00a41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:06.688537 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf5ebd-7d81-4465-9e25-6ca1ae00a41a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:06.690430 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf5ebd-7d81-4465-9e25-6ca1ae00a41a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:06.690924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf5ebd-7d81-4465-9e25-6ca1ae00a41a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:06.691698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf5ebd-7d81-4465-9e25-6ca1ae00a41a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:06.694919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf5ebd-7d81-4465-9e25-6ca1ae00a41a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-bdaf5ebd-7d81-4465-9e25-6ca1ae00a41a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:06.709063 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:06.710281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:06.716223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 362/710] 10.222.0.33 () {56 vars in 1129 bytes} [Tue Oct 4 12:36:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:06.717127 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf5ebd-7d81-4465-9e25-6ca1ae00a41a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:06.790939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a428c0-d6f0-4f9f-977b-ac32595a0e48 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:06.833430 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c090e31-e1c7-4e8e-a9cb-c53a34be28f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:06.834463 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c090e31-e1c7-4e8e-a9cb-c53a34be28f4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:06.834925 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c090e31-e1c7-4e8e-a9cb-c53a34be28f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:06.835378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c090e31-e1c7-4e8e-a9cb-c53a34be28f4 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:06.836243 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c090e31-e1c7-4e8e-a9cb-c53a34be28f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:06.836724 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c090e31-e1c7-4e8e-a9cb-c53a34be28f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:06.838484 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c090e31-e1c7-4e8e-a9cb-c53a34be28f4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:06.853540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 349/711] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:06.854875 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c090e31-e1c7-4e8e-a9cb-c53a34be28f4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:06.880243 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf5ebd-7d81-4465-9e25-6ca1ae00a41a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:06.895062 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:06.906517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:06.941728 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c60b1-471d-446e-9f52-91cb45cdc617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:06.942696 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c60b1-471d-446e-9f52-91cb45cdc617 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:06.943147 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c60b1-471d-446e-9f52-91cb45cdc617 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:06.943594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c60b1-471d-446e-9f52-91cb45cdc617 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:06.959794 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c60b1-471d-446e-9f52-91cb45cdc617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-8b7c60b1-471d-446e-9f52-91cb45cdc617 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:06.966464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:06.968472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 363/712] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:36:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:06.971497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c60b1-471d-446e-9f52-91cb45cdc617 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:06.999351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c090e31-e1c7-4e8e-a9cb-c53a34be28f4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:07.054782 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c394b6-e1ac-4573-a71f-be3847031032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:07.055419 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c394b6-e1ac-4573-a71f-be3847031032 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:07.055742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c394b6-e1ac-4573-a71f-be3847031032 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:07.056015 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c394b6-e1ac-4573-a71f-be3847031032 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:07.057269 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c394b6-e1ac-4573-a71f-be3847031032 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:07.057729 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c394b6-e1ac-4573-a71f-be3847031032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:07.058997 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c394b6-e1ac-4573-a71f-be3847031032 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:07.072373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 350/713] 10.222.0.33 () {56 vars in 1124 bytes} [Tue Oct 4 12:36:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:07.072373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ecc2c-4f27-43b1-9468-17d37cc55022 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:07.072373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ecc2c-4f27-43b1-9468-17d37cc55022 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:07.072373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ecc2c-4f27-43b1-9468-17d37cc55022 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:07.072373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 351/714] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:07.072373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ecc2c-4f27-43b1-9468-17d37cc55022 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:07.155445 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c60b1-471d-446e-9f52-91cb45cdc617 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:07.187748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:07.210647 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ecc2c-4f27-43b1-9468-17d37cc55022 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:07.217810 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25e79e-d16f-4dab-82c1-c5584e5b52cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:07.218897 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25e79e-d16f-4dab-82c1-c5584e5b52cb None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:07.219371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25e79e-d16f-4dab-82c1-c5584e5b52cb None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:07.219843 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25e79e-d16f-4dab-82c1-c5584e5b52cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:07.232495 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25e79e-d16f-4dab-82c1-c5584e5b52cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:07.235248 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25e79e-d16f-4dab-82c1-c5584e5b52cb None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-7a25e79e-d16f-4dab-82c1-c5584e5b52cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:07.260413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:07.261642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:07.262854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:07.262854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:07.262854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:07.262854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:07.262854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:07.262854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:07.262854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:07.262854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:07.262854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 364/715] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:07.264884 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25e79e-d16f-4dab-82c1-c5584e5b52cb None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:07.274989 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0682a-4f9a-4925-9481-e5718f3e6fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:07.275796 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0682a-4f9a-4925-9481-e5718f3e6fc9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:07.276203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0682a-4f9a-4925-9481-e5718f3e6fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:07.276610 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0682a-4f9a-4925-9481-e5718f3e6fc9 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:07.277821 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0682a-4f9a-4925-9481-e5718f3e6fc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:07.278315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0682a-4f9a-4925-9481-e5718f3e6fc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:07.280259 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0682a-4f9a-4925-9481-e5718f3e6fc9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:07.300186 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 352/716] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:36:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:07.300186 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7014b68-1eaa-4988-a2bd-8cee7e8e845b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:07.300186 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7014b68-1eaa-4988-a2bd-8cee7e8e845b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:07.300186 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7014b68-1eaa-4988-a2bd-8cee7e8e845b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:07.307917 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:07.322713 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c7014b68-1eaa-4988-a2bd-8cee7e8e845b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:07.324494 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c7014b68-1eaa-4988-a2bd-8cee7e8e845b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:07.399601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25e79e-d16f-4dab-82c1-c5584e5b52cb None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:07.434069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 353/717] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:36:07.434069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7014b68-1eaa-4988-a2bd-8cee7e8e845b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:07.466836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206983d-dddf-45da-ad15-7d9b8b988dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:07.467529 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206983d-dddf-45da-ad15-7d9b8b988dd7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:07.467987 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206983d-dddf-45da-ad15-7d9b8b988dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:07.468492 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206983d-dddf-45da-ad15-7d9b8b988dd7 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:07.469536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206983d-dddf-45da-ad15-7d9b8b988dd7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:07.470064 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206983d-dddf-45da-ad15-7d9b8b988dd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:07.472671 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206983d-dddf-45da-ad15-7d9b8b988dd7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:07.490600 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 365/718] 10.222.0.33 () {56 vars in 1124 bytes} [Tue Oct 4 12:36:07 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:07.491593 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206983d-dddf-45da-ad15-7d9b8b988dd7 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:07.588976 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:07.638763 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7014b68-1eaa-4988-a2bd-8cee7e8e845b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:07.663287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:07.680568 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206983d-dddf-45da-ad15-7d9b8b988dd7 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:07.705294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af24c0e-244a-434b-82eb-e510f78327bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:07.706383 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af24c0e-244a-434b-82eb-e510f78327bf None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:07.706928 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af24c0e-244a-434b-82eb-e510f78327bf None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:07.707424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af24c0e-244a-434b-82eb-e510f78327bf None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:07.708330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af24c0e-244a-434b-82eb-e510f78327bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:07.708841 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af24c0e-244a-434b-82eb-e510f78327bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:07.710716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af24c0e-244a-434b-82eb-e510f78327bf None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:07.727482 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 354/719] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:07.728770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c153990-d24d-49d6-970d-89c7f9b9194a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:07.729418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c153990-d24d-49d6-970d-89c7f9b9194a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:07.729696 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c153990-d24d-49d6-970d-89c7f9b9194a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:07.729966 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c153990-d24d-49d6-970d-89c7f9b9194a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:07.736475 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af24c0e-244a-434b-82eb-e510f78327bf None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:07.758249 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c153990-d24d-49d6-970d-89c7f9b9194a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:07.760762 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c153990-d24d-49d6-970d-89c7f9b9194a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-2c153990-d24d-49d6-970d-89c7f9b9194a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:07.793086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:07.794133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:07.795023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:07.795023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:07.795023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:07.804279 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 366/720] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:07.809810 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c153990-d24d-49d6-970d-89c7f9b9194a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:07.869427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af24c0e-244a-434b-82eb-e510f78327bf None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:07.910394 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19bdf8-e28a-46c0-ac65-5f751eb8f059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:07.911011 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19bdf8-e28a-46c0-ac65-5f751eb8f059 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:07.911286 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19bdf8-e28a-46c0-ac65-5f751eb8f059 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:07.911555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19bdf8-e28a-46c0-ac65-5f751eb8f059 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:07.912512 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19bdf8-e28a-46c0-ac65-5f751eb8f059 None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=02260480264b4f8f8a7e5cb647ac5d9e) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:07.912822 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19bdf8-e28a-46c0-ac65-5f751eb8f059 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:07.914305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19bdf8-e28a-46c0-ac65-5f751eb8f059 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:07.921231 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 355/721] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:36:07 2022] GET /identity/v3/role_assignments?user.id=02260480264b4f8f8a7e5cb647ac5d9e&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=fc68d072459d4720b9666c93bd535d63 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:07.926479 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c153990-d24d-49d6-970d-89c7f9b9194a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:07.930411 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19bdf8-e28a-46c0-ac65-5f751eb8f059 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:07.934507 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:07.970027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65154168-e3f7-487d-bbf0-f6c5efa64b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:07.971181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65154168-e3f7-487d-bbf0-f6c5efa64b89 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:07.971893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65154168-e3f7-487d-bbf0-f6c5efa64b89 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:07.972374 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65154168-e3f7-487d-bbf0-f6c5efa64b89 None admin] PATH_INFO: `/v3/services/network` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:07.974570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65154168-e3f7-487d-bbf0-f6c5efa64b89 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:07.976569 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65154168-e3f7-487d-bbf0-f6c5efa64b89 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-65154168-e3f7-487d-bbf0-f6c5efa64b89 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:07.993287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:07.994401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 12:36:07.995221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:08.003255 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 367/722] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:07 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:08.012016 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65154168-e3f7-487d-bbf0-f6c5efa64b89 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:08.026327 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:08.193231 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19bdf8-e28a-46c0-ac65-5f751eb8f059 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:08.251190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65154168-e3f7-487d-bbf0-f6c5efa64b89 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:08.277750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ccbe8-56c8-4148-9436-e886d115d054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:08.279267 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ccbe8-56c8-4148-9436-e886d115d054 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:08.280447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ccbe8-56c8-4148-9436-e886d115d054 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:08.281245 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ccbe8-56c8-4148-9436-e886d115d054 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:08.282779 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ccbe8-56c8-4148-9436-e886d115d054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:08.283753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ccbe8-56c8-4148-9436-e886d115d054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:08.286669 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ccbe8-56c8-4148-9436-e886d115d054 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:08.313454 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 356/723] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:08.319207 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ccbe8-56c8-4148-9436-e886d115d054 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:08.364697 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd382da-d335-4dfa-a018-07383ba604e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:08.365332 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd382da-d335-4dfa-a018-07383ba604e9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:08.365609 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd382da-d335-4dfa-a018-07383ba604e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:08.365877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd382da-d335-4dfa-a018-07383ba604e9 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:08.366550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd382da-d335-4dfa-a018-07383ba604e9 None admin] RBAC: Adding query filter params (name=network) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:08.366851 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd382da-d335-4dfa-a018-07383ba604e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:08.368274 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd382da-d335-4dfa-a018-07383ba604e9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:08.381379 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 368/724] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:08.407172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd382da-d335-4dfa-a018-07383ba604e9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:08.485174 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:08.564182 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ccbe8-56c8-4148-9436-e886d115d054 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:08.594317 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:08.651607 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f61aa9-8af4-4af8-bb20-331054127cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:08.653859 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61aa9-8af4-4af8-bb20-331054127cee None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:08.654748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61aa9-8af4-4af8-bb20-331054127cee None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:08.655915 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61aa9-8af4-4af8-bb20-331054127cee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:08.673855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f61aa9-8af4-4af8-bb20-331054127cee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:08.677185 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f61aa9-8af4-4af8-bb20-331054127cee None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:08.685240 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd382da-d335-4dfa-a018-07383ba604e9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c8f61aa9-8af4-4af8-bb20-331054127cee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:08.711315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:08.713848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:08.717362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:08.717362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:08.717362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:08.717362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:08.717362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:08.717362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:08.717362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:08.717362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:08.717362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 12:36:08.717362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:08.722407 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 357/725] 10.222.0.33 () {56 vars in 1117 bytes} [Tue Oct 4 12:36:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:08.731141 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f61aa9-8af4-4af8-bb20-331054127cee None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:08.773463 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832616cd-d443-4ce2-9c1c-35f682fcbb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:08.776609 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832616cd-d443-4ce2-9c1c-35f682fcbb38 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:08.777489 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832616cd-d443-4ce2-9c1c-35f682fcbb38 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:08.778695 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832616cd-d443-4ce2-9c1c-35f682fcbb38 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:08.780160 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832616cd-d443-4ce2-9c1c-35f682fcbb38 None admin] RBAC: Adding query filter params (type=network) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:08.781061 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832616cd-d443-4ce2-9c1c-35f682fcbb38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:08.784679 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832616cd-d443-4ce2-9c1c-35f682fcbb38 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:08.795675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 369/726] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:08 2022] GET /identity/v3/services?type=network => generated 124 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:08.946572 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f61aa9-8af4-4af8-bb20-331054127cee None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:08.982510 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:08.998382 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec7ec8-2b5a-44fd-a8f7-d421bc312b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:08.999015 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec7ec8-2b5a-44fd-a8f7-d421bc312b25 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:08.999291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec7ec8-2b5a-44fd-a8f7-d421bc312b25 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:08.999560 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec7ec8-2b5a-44fd-a8f7-d421bc312b25 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:09.000316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec7ec8-2b5a-44fd-a8f7-d421bc312b25 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:09.000621 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec7ec8-2b5a-44fd-a8f7-d421bc312b25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:09.002143 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec7ec8-2b5a-44fd-a8f7-d421bc312b25 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:09.034230 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 358/727] 10.222.0.33 () {56 vars in 1149 bytes} [Tue Oct 4 12:36:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:09.040891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832616cd-d443-4ce2-9c1c-35f682fcbb38 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:09.234975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832616cd-d443-4ce2-9c1c-35f682fcbb38 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:09.320273 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a2abf-d981-4e28-938d-04034181b03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:09.320892 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a2abf-d981-4e28-938d-04034181b03c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:09.321171 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a2abf-d981-4e28-938d-04034181b03c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:09.321453 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a2abf-d981-4e28-938d-04034181b03c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:09.329998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:09.335967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4d548-8181-4fcd-9e4a-d4cf3ff0daee None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:09.335967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4d548-8181-4fcd-9e4a-d4cf3ff0daee None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:09.336346 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4d548-8181-4fcd-9e4a-d4cf3ff0daee None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:09.337913 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 359/728] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:36:09.343735 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa5f29-636b-4ed2-9a13-718d943224f1 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:09.344133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa5f29-636b-4ed2-9a13-718d943224f1 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:09.344291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa5f29-636b-4ed2-9a13-718d943224f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:09.371587 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9efa5f29-636b-4ed2-9a13-718d943224f1 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:09.372830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9efa5f29-636b-4ed2-9a13-718d943224f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:09.461187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a2abf-d981-4e28-938d-04034181b03c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:09.462788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a2abf-d981-4e28-938d-04034181b03c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:09.508496 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 360/729] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:36:09.685812 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 370/730] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:36:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:36:09.691116 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa5f29-636b-4ed2-9a13-718d943224f1 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:09.864735 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa5f29-636b-4ed2-9a13-718d943224f1 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:09.903025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:09.946025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b6181-aa12-4f4f-bfc2-fb88e069e5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:09.947695 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b6181-aa12-4f4f-bfc2-fb88e069e5ef None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:09.948182 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b6181-aa12-4f4f-bfc2-fb88e069e5ef None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:09.948669 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b6181-aa12-4f4f-bfc2-fb88e069e5ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:09.952886 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a2abf-d981-4e28-938d-04034181b03c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:09.966675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b6181-aa12-4f4f-bfc2-fb88e069e5ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:09.969564 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b6181-aa12-4f4f-bfc2-fb88e069e5ef None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-008b6181-aa12-4f4f-bfc2-fb88e069e5ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:10.016908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:10.018752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:10.020708 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:10.020708 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:10.020708 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:10.021683 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 361/731] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:10.025164 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b6181-aa12-4f4f-bfc2-fb88e069e5ef None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:10.074488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:10.326075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a2abf-d981-4e28-938d-04034181b03c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:10.403553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b6181-aa12-4f4f-bfc2-fb88e069e5ef None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:10.406358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a64ee-6c19-45ff-9d5d-d64db1d29bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:10.407084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a64ee-6c19-45ff-9d5d-d64db1d29bbe None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:10.407359 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a64ee-6c19-45ff-9d5d-d64db1d29bbe None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:10.407659 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a64ee-6c19-45ff-9d5d-d64db1d29bbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:10.420912 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a64ee-6c19-45ff-9d5d-d64db1d29bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-391a64ee-6c19-45ff-9d5d-d64db1d29bbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:10.426130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:10.427919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 371/732] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:36:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 475 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:10.429617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e863f-716e-4d12-9a31-168c56f3a3c9 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:10.434443 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e863f-716e-4d12-9a31-168c56f3a3c9 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:10.434443 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e863f-716e-4d12-9a31-168c56f3a3c9 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:10.434443 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 372/733] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:10.438082 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb52f9-94e8-4d0a-8a9d-d465168642e9 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:10.438082 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb52f9-94e8-4d0a-8a9d-d465168642e9 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:10.438082 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb52f9-94e8-4d0a-8a9d-d465168642e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:10.460562 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-82fb52f9-94e8-4d0a-8a9d-d465168642e9 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:10.462134 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-82fb52f9-94e8-4d0a-8a9d-d465168642e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:10.468223 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd055a5a-82ca-4506-a91f-621604e645fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:10.468965 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd055a5a-82ca-4506-a91f-621604e645fd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:10.469373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd055a5a-82ca-4506-a91f-621604e645fd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:10.469679 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd055a5a-82ca-4506-a91f-621604e645fd None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:10.470609 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd055a5a-82ca-4506-a91f-621604e645fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:10.470964 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd055a5a-82ca-4506-a91f-621604e645fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:10.473614 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd055a5a-82ca-4506-a91f-621604e645fd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:10.494262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 362/734] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 470 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:10.495407 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd055a5a-82ca-4506-a91f-621604e645fd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:10.536297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:10.570066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 373/735] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 12:36:10.570066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb52f9-94e8-4d0a-8a9d-d465168642e9 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:10.621098 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:10.731534 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd055a5a-82ca-4506-a91f-621604e645fd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:10.774607 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb52f9-94e8-4d0a-8a9d-d465168642e9 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:10.811532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45850f2-d36f-44c5-a2ea-3757179fb0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:10.814144 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45850f2-d36f-44c5-a2ea-3757179fb0cb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:10.815129 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45850f2-d36f-44c5-a2ea-3757179fb0cb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:10.815981 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45850f2-d36f-44c5-a2ea-3757179fb0cb None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:10.817286 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45850f2-d36f-44c5-a2ea-3757179fb0cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:10.817786 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45850f2-d36f-44c5-a2ea-3757179fb0cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:10.820063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45850f2-d36f-44c5-a2ea-3757179fb0cb None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:10.821460 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a46ee-222a-47c7-947f-6a07b1e3e7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:10.823309 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a46ee-222a-47c7-947f-6a07b1e3e7e5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:10.823744 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a46ee-222a-47c7-947f-6a07b1e3e7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:10.824182 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a46ee-222a-47c7-947f-6a07b1e3e7e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:10.834408 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a46ee-222a-47c7-947f-6a07b1e3e7e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:10.836173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a46ee-222a-47c7-947f-6a07b1e3e7e5 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:10.841772 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 363/736] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:36:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:10.841772 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9addf9-ddc2-478e-aa86-d5d35d901ae7 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:10.841772 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9addf9-ddc2-478e-aa86-d5d35d901ae7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:10.841772 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9addf9-ddc2-478e-aa86-d5d35d901ae7 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:10.842931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 364/737] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:10.846082 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9addf9-ddc2-478e-aa86-d5d35d901ae7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-b21a46ee-222a-47c7-947f-6a07b1e3e7e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:10.861898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:10.863190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:10.864995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:10.864995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:10.864995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:10.864995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:10.864995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:10.864995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:10.864995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:10.864995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:10.868214 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 374/738] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:10.869881 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8394d-084d-4fe0-93af-a7dcf0d4706c None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:10.870390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8394d-084d-4fe0-93af-a7dcf0d4706c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:10.870865 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8394d-084d-4fe0-93af-a7dcf0d4706c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:10.896235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-8aa8394d-084d-4fe0-93af-a7dcf0d4706c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:10.898448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-8aa8394d-084d-4fe0-93af-a7dcf0d4706c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:10.989850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:10.999772 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 375/739] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:36:11.001661 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8394d-084d-4fe0-93af-a7dcf0d4706c None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:11.010758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9addf9-ddc2-478e-aa86-d5d35d901ae7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:11.034898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:11.060258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf6ad5-91ec-4251-9e99-5e1f2ac643d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:11.060986 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf6ad5-91ec-4251-9e99-5e1f2ac643d2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:11.061272 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf6ad5-91ec-4251-9e99-5e1f2ac643d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:11.061602 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf6ad5-91ec-4251-9e99-5e1f2ac643d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:11.074911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf6ad5-91ec-4251-9e99-5e1f2ac643d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:11.076440 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf6ad5-91ec-4251-9e99-5e1f2ac643d2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-20cf6ad5-91ec-4251-9e99-5e1f2ac643d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:11.095857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:11.096890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:11.097795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:11.097795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:11.097795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:11.102483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 365/740] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:11.103812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf6ad5-91ec-4251-9e99-5e1f2ac643d2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:11.173175 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8394d-084d-4fe0-93af-a7dcf0d4706c None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:11.235627 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a765a6-0878-4d02-b1dc-a648bf5e1c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:11.237184 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a765a6-0878-4d02-b1dc-a648bf5e1c22 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:11.237690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a765a6-0878-4d02-b1dc-a648bf5e1c22 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:11.238006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a765a6-0878-4d02-b1dc-a648bf5e1c22 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:11.239663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a765a6-0878-4d02-b1dc-a648bf5e1c22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:11.240449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a765a6-0878-4d02-b1dc-a648bf5e1c22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:11.244594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a765a6-0878-4d02-b1dc-a648bf5e1c22 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:11.248720 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf6ad5-91ec-4251-9e99-5e1f2ac643d2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:11.281264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 376/741] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:11.281264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a765a6-0878-4d02-b1dc-a648bf5e1c22 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:11.321379 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cba7c-2f5f-46b9-a374-4b2e823f3405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:11.322511 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cba7c-2f5f-46b9-a374-4b2e823f3405 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:11.323047 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cba7c-2f5f-46b9-a374-4b2e823f3405 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:11.323649 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cba7c-2f5f-46b9-a374-4b2e823f3405 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:11.324838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cba7c-2f5f-46b9-a374-4b2e823f3405 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:11.327329 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cba7c-2f5f-46b9-a374-4b2e823f3405 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:11.347458 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:11.360421 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 366/742] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:36:11 2022] POST /identity/v3/services => generated 251 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:36:11.362114 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cba7c-2f5f-46b9-a374-4b2e823f3405 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:11.459743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:11.515023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a765a6-0878-4d02-b1dc-a648bf5e1c22 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:11.577360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda621-3619-42ac-a3bb-806daad60148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:11.578333 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda621-3619-42ac-a3bb-806daad60148 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:11.578794 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda621-3619-42ac-a3bb-806daad60148 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:11.579262 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda621-3619-42ac-a3bb-806daad60148 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:11.580116 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eda621-3619-42ac-a3bb-806daad60148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:11.580620 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eda621-3619-42ac-a3bb-806daad60148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:11.582283 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eda621-3619-42ac-a3bb-806daad60148 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:11.598366 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 377/743] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:11.599690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda621-3619-42ac-a3bb-806daad60148 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:11.600432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cba7c-2f5f-46b9-a374-4b2e823f3405 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:11.676672 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc5f36-e4e4-4fd6-be69-1ce04b1495dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:11.677617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc5f36-e4e4-4fd6-be69-1ce04b1495dc None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:11.678162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc5f36-e4e4-4fd6-be69-1ce04b1495dc None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:11.678611 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc5f36-e4e4-4fd6-be69-1ce04b1495dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:11.679555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc5f36-e4e4-4fd6-be69-1ce04b1495dc None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=58d82737c2864faf8fef3b53167c4dfc) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:11.680101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc5f36-e4e4-4fd6-be69-1ce04b1495dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:11.682520 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc5f36-e4e4-4fd6-be69-1ce04b1495dc None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:11.689029 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 367/744] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:36:11 2022] GET /identity/v3/role_assignments?user.id=58d82737c2864faf8fef3b53167c4dfc&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 259 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:11.690439 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc5f36-e4e4-4fd6-be69-1ce04b1495dc None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:11.793863 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda621-3619-42ac-a3bb-806daad60148 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:11.850385 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:11.887342 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:11.897790 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60db28-6d66-4e5d-ba5a-a26cf2846213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:11.901595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60db28-6d66-4e5d-ba5a-a26cf2846213 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:11.902516 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60db28-6d66-4e5d-ba5a-a26cf2846213 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:11.903205 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60db28-6d66-4e5d-ba5a-a26cf2846213 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:11.924367 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60db28-6d66-4e5d-ba5a-a26cf2846213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-2d60db28-6d66-4e5d-ba5a-a26cf2846213 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:11.933513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:11.935298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:11.936792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:11.936792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:11.936792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:11.936792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:11.936792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:11.936792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:11.936792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:11.936792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:11.936792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:11.936792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:11.936792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:36:11.936792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:11.941067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 378/745] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:36:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:11.953736 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60db28-6d66-4e5d-ba5a-a26cf2846213 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:11.955098 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc5f36-e4e4-4fd6-be69-1ce04b1495dc None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:11.993210 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d5e1a-8f54-43a0-be27-c0ddb72902a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:11.993890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d5e1a-8f54-43a0-be27-c0ddb72902a4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:11.994191 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d5e1a-8f54-43a0-be27-c0ddb72902a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:11.994462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d5e1a-8f54-43a0-be27-c0ddb72902a4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:12.003668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d5e1a-8f54-43a0-be27-c0ddb72902a4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:12.005137 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d5e1a-8f54-43a0-be27-c0ddb72902a4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-693d5e1a-8f54-43a0-be27-c0ddb72902a4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:12.018536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:12.021177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:12.023078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:12.023078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:12.023078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:12.023078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:12.023078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:12.023078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:12.023078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:12.023078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:12.023078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 12:36:12.023078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:12.023078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 368/746] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:36:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:12.030070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d5e1a-8f54-43a0-be27-c0ddb72902a4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:12.138415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60db28-6d66-4e5d-ba5a-a26cf2846213 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:12.318816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597de92-407a-463c-a8f6-7b90ddc14542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:12.319731 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597de92-407a-463c-a8f6-7b90ddc14542 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:12.320017 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597de92-407a-463c-a8f6-7b90ddc14542 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:12.320292 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597de92-407a-463c-a8f6-7b90ddc14542 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:12.321449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597de92-407a-463c-a8f6-7b90ddc14542 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:12.322314 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597de92-407a-463c-a8f6-7b90ddc14542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:12.324067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597de92-407a-463c-a8f6-7b90ddc14542 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:12.340760 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 379/747] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:36:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:12.345906 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d5e1a-8f54-43a0-be27-c0ddb72902a4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:12.347795 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597de92-407a-463c-a8f6-7b90ddc14542 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:12.419635 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:12.449140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ba0e0-215e-489a-9437-19cd361d06d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:12.450725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ba0e0-215e-489a-9437-19cd361d06d8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:12.451489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ba0e0-215e-489a-9437-19cd361d06d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:12.452235 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ba0e0-215e-489a-9437-19cd361d06d8 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:12.453869 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ba0e0-215e-489a-9437-19cd361d06d8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:12.454701 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ba0e0-215e-489a-9437-19cd361d06d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:12.458525 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ba0e0-215e-489a-9437-19cd361d06d8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:12.499609 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 369/748] 10.222.0.33 () {56 vars in 1135 bytes} [Tue Oct 4 12:36:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:12.506228 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ba0e0-215e-489a-9437-19cd361d06d8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:12.524308 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:12.586416 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597de92-407a-463c-a8f6-7b90ddc14542 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:12.631407 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3839cc-e0e9-48d8-8070-2edca6ce7153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:12.632055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3839cc-e0e9-48d8-8070-2edca6ce7153 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:12.632335 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3839cc-e0e9-48d8-8070-2edca6ce7153 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:12.632595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3839cc-e0e9-48d8-8070-2edca6ce7153 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:12.648393 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3839cc-e0e9-48d8-8070-2edca6ce7153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:12.649933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3839cc-e0e9-48d8-8070-2edca6ce7153 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-aa3839cc-e0e9-48d8-8070-2edca6ce7153 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:12.677705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:12.679570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:12.681739 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:12.681739 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:12.681739 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:12.684984 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 380/749] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:12.687400 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4001b-1600-4dc7-ba33-67b5234d2b7c None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:12.688267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4001b-1600-4dc7-ba33-67b5234d2b7c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:12.689117 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4001b-1600-4dc7-ba33-67b5234d2b7c None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:12.692100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 381/750] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:12.693578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4001b-1600-4dc7-ba33-67b5234d2b7c None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:12.708779 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ba0e0-215e-489a-9437-19cd361d06d8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:12.752908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1495e-895f-4fb6-9ad2-f597526bedbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:12.753539 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1495e-895f-4fb6-9ad2-f597526bedbd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:12.753811 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1495e-895f-4fb6-9ad2-f597526bedbd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:12.754109 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1495e-895f-4fb6-9ad2-f597526bedbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:12.810500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:12.840563 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1495e-895f-4fb6-9ad2-f597526bedbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:12.842070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1495e-895f-4fb6-9ad2-f597526bedbd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:12.888111 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4001b-1600-4dc7-ba33-67b5234d2b7c None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:12.903301 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:12.953660 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52db64b-cb83-4e1a-b23b-a4a07e53138b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:12.955134 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52db64b-cb83-4e1a-b23b-a4a07e53138b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:12.955942 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52db64b-cb83-4e1a-b23b-a4a07e53138b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:12.956770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52db64b-cb83-4e1a-b23b-a4a07e53138b None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:12.958228 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52db64b-cb83-4e1a-b23b-a4a07e53138b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:12.959053 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52db64b-cb83-4e1a-b23b-a4a07e53138b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:12.961964 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52db64b-cb83-4e1a-b23b-a4a07e53138b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:12.970253 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 370/751] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:36:12 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 12:36:12.973444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc16b2-eb7b-4d39-bbd5-662461406a90 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:12.973738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc16b2-eb7b-4d39-bbd5-662461406a90 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:12.974008 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc16b2-eb7b-4d39-bbd5-662461406a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:12.995972 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9cfc16b2-eb7b-4d39-bbd5-662461406a90 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:12.996522 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 382/752] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:12.997149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9cfc16b2-eb7b-4d39-bbd5-662461406a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:12.998691 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52db64b-cb83-4e1a-b23b-a4a07e53138b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:13.086202 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 371/753] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 12:36:13.087261 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc16b2-eb7b-4d39-bbd5-662461406a90 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:13.108190 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:13.242180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52db64b-cb83-4e1a-b23b-a4a07e53138b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:13.245010 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc16b2-eb7b-4d39-bbd5-662461406a90 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:13.321771 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4af22-d2d7-4de7-a554-98799c13bca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:13.322453 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4af22-d2d7-4de7-a554-98799c13bca0 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:13.322730 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4af22-d2d7-4de7-a554-98799c13bca0 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:13.323004 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4af22-d2d7-4de7-a554-98799c13bca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:13.328526 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6462854-e13b-44a8-8b85-7119cba7978b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:13.329181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6462854-e13b-44a8-8b85-7119cba7978b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:13.329465 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6462854-e13b-44a8-8b85-7119cba7978b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:13.329735 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6462854-e13b-44a8-8b85-7119cba7978b None admin] PATH_INFO: `/v3/users/nova` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:13.347957 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6462854-e13b-44a8-8b85-7119cba7978b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:13.349451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6462854-e13b-44a8-8b85-7119cba7978b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:13.350754 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4af22-d2d7-4de7-a554-98799c13bca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:13.352207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4af22-d2d7-4de7-a554-98799c13bca0 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-f6462854-e13b-44a8-8b85-7119cba7978b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:13.374884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:13.378121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:13.382574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:13.382574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:13.382574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:13.382574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:13.382574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:13.382574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:13.382574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:13.382574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:13.382574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:36:13.382574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:13.382574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 372/754] 10.222.0.33 () {56 vars in 1093 bytes} [Tue Oct 4 12:36:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:13.386841 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6462854-e13b-44a8-8b85-7119cba7978b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-37e4af22-d2d7-4de7-a554-98799c13bca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:13.392115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:13.394626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:13.396762 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:13.396762 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:13.396762 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:13.396762 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 383/755] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:13.402318 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4af22-d2d7-4de7-a554-98799c13bca0 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:13.450064 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:13.562498 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:13.642088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4af22-d2d7-4de7-a554-98799c13bca0 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:13.677321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6462854-e13b-44a8-8b85-7119cba7978b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:13.708885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914deb05-bc94-48c9-b5f0-aa3c20d7ca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:13.709506 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914deb05-bc94-48c9-b5f0-aa3c20d7ca38 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:13.709781 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914deb05-bc94-48c9-b5f0-aa3c20d7ca38 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:13.710075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914deb05-bc94-48c9-b5f0-aa3c20d7ca38 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:13.710829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914deb05-bc94-48c9-b5f0-aa3c20d7ca38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:13.711135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914deb05-bc94-48c9-b5f0-aa3c20d7ca38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:13.712610 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914deb05-bc94-48c9-b5f0-aa3c20d7ca38 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:13.732103 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 384/756] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:13.733097 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914deb05-bc94-48c9-b5f0-aa3c20d7ca38 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:13.759789 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107e625-3869-42dd-8464-0ebb5e1674a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:13.761609 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107e625-3869-42dd-8464-0ebb5e1674a3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:13.762398 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107e625-3869-42dd-8464-0ebb5e1674a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:13.762826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107e625-3869-42dd-8464-0ebb5e1674a3 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:13.764358 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107e625-3869-42dd-8464-0ebb5e1674a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:13.765517 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107e625-3869-42dd-8464-0ebb5e1674a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:13.769573 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107e625-3869-42dd-8464-0ebb5e1674a3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:13.809911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 373/757] 10.222.0.33 () {56 vars in 1133 bytes} [Tue Oct 4 12:36:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:13.811413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107e625-3869-42dd-8464-0ebb5e1674a3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:13.865590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:13.910398 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914deb05-bc94-48c9-b5f0-aa3c20d7ca38 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:13.978454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371681d-089b-4af9-9812-e51a0b18c4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:13.979816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371681d-089b-4af9-9812-e51a0b18c4f4 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:13.980539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371681d-089b-4af9-9812-e51a0b18c4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:13.981159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371681d-089b-4af9-9812-e51a0b18c4f4 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:13.982460 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d371681d-089b-4af9-9812-e51a0b18c4f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:13.984946 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d371681d-089b-4af9-9812-e51a0b18c4f4 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:14.011412 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107e625-3869-42dd-8464-0ebb5e1674a3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:14.015304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 385/758] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:36:13 2022] POST /identity/v3/services => generated 240 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:36:14.017444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371681d-089b-4af9-9812-e51a0b18c4f4 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:14.059710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:14.123029 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e1c6e-203c-4a2f-8839-201080811e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:14.123604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e1c6e-203c-4a2f-8839-201080811e3d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:14.123887 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e1c6e-203c-4a2f-8839-201080811e3d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:14.124195 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e1c6e-203c-4a2f-8839-201080811e3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:14.132325 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e1c6e-203c-4a2f-8839-201080811e3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:14.133903 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e1c6e-203c-4a2f-8839-201080811e3d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-7b1e1c6e-203c-4a2f-8839-201080811e3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:14.150515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:14.152435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:14.154316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:14.154316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:14.154316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:14.154316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:14.154316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:14.154316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:14.154316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:14.154316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:14.154316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 374/759] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:14.154316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454a7ac-f3cd-4b51-b2d1-eb7e7fa663ef None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:14.154316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454a7ac-f3cd-4b51-b2d1-eb7e7fa663ef None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:14.155633 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454a7ac-f3cd-4b51-b2d1-eb7e7fa663ef None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:14.160928 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 375/760] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:14.161840 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454a7ac-f3cd-4b51-b2d1-eb7e7fa663ef None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:14.247073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371681d-089b-4af9-9812-e51a0b18c4f4 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:14.300960 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454a7ac-f3cd-4b51-b2d1-eb7e7fa663ef None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:14.319975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ba221-5417-42d5-8039-d89c123ff49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:14.319975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ba221-5417-42d5-8039-d89c123ff49f None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:14.319975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ba221-5417-42d5-8039-d89c123ff49f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:14.319975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ba221-5417-42d5-8039-d89c123ff49f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:14.330481 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:14.332177 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ba221-5417-42d5-8039-d89c123ff49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:14.333426 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ba221-5417-42d5-8039-d89c123ff49f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-f85ba221-5417-42d5-8039-d89c123ff49f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:14.355686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:14.357420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:14.359169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:14.359169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:14.359169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:14.359169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 386/761] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:14.359169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8be2e-5d9c-46ec-941d-cded2a428660 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:14.359169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8be2e-5d9c-46ec-941d-cded2a428660 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:14.359169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8be2e-5d9c-46ec-941d-cded2a428660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:14.373766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c769c-8e73-42ae-b3ad-fce760020ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:14.375480 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c769c-8e73-42ae-b3ad-fce760020ecf None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:14.376235 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c769c-8e73-42ae-b3ad-fce760020ecf None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:14.377024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c769c-8e73-42ae-b3ad-fce760020ecf None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:14.378829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c769c-8e73-42ae-b3ad-fce760020ecf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:14.379435 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-4cb8be2e-5d9c-46ec-941d-cded2a428660 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:14.380016 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c769c-8e73-42ae-b3ad-fce760020ecf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:14.380673 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-4cb8be2e-5d9c-46ec-941d-cded2a428660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:14.383428 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c769c-8e73-42ae-b3ad-fce760020ecf None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:14.413354 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 376/762] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:14.415970 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c769c-8e73-42ae-b3ad-fce760020ecf None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:14.483277 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:14.489405 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 387/763] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:14.490678 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8be2e-5d9c-46ec-941d-cded2a428660 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:14.612985 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8be2e-5d9c-46ec-941d-cded2a428660 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:14.638899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:14.665156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a9a12-510a-4cfa-b847-1341059b7381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:14.666136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a9a12-510a-4cfa-b847-1341059b7381 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:14.666555 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a9a12-510a-4cfa-b847-1341059b7381 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:14.666974 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a9a12-510a-4cfa-b847-1341059b7381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:14.669242 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a9a12-510a-4cfa-b847-1341059b7381 None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=332fdff1d02a4664923ddbe0478eaa7a) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:14.669755 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a9a12-510a-4cfa-b847-1341059b7381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:14.672300 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a9a12-510a-4cfa-b847-1341059b7381 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:14.681501 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c769c-8e73-42ae-b3ad-fce760020ecf None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:14.691763 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 388/764] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:36:14 2022] GET /identity/v3/role_assignments?user.id=332fdff1d02a4664923ddbe0478eaa7a&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=fc68d072459d4720b9666c93bd535d63 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:14.744597 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d52814-fbc9-4203-a3f2-5eb510bfb5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:14.745523 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d52814-fbc9-4203-a3f2-5eb510bfb5e0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:14.745948 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d52814-fbc9-4203-a3f2-5eb510bfb5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:14.746389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d52814-fbc9-4203-a3f2-5eb510bfb5e0 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:14.747192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d52814-fbc9-4203-a3f2-5eb510bfb5e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:14.747641 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d52814-fbc9-4203-a3f2-5eb510bfb5e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:14.749205 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d52814-fbc9-4203-a3f2-5eb510bfb5e0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:14.767296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 377/765] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:14.776456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a9a12-510a-4cfa-b847-1341059b7381 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:14.908129 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916b4a3-dd84-488d-ab04-ccc5c9919c2a None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:14.908668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916b4a3-dd84-488d-ab04-ccc5c9919c2a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:14.909124 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916b4a3-dd84-488d-ab04-ccc5c9919c2a None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:14.910763 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 378/766] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:14.923904 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce3dce-5929-4d85-a237-1bf3284460d2 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:14.924177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce3dce-5929-4d85-a237-1bf3284460d2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:14.924474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce3dce-5929-4d85-a237-1bf3284460d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:14.926954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a9a12-510a-4cfa-b847-1341059b7381 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:14.971671 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4dce3dce-5929-4d85-a237-1bf3284460d2 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:14.973849 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4dce3dce-5929-4d85-a237-1bf3284460d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:14.981009 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7989a8b-e7e3-4a76-b030-f0db68373281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:14.981948 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7989a8b-e7e3-4a76-b030-f0db68373281 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:14.982442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7989a8b-e7e3-4a76-b030-f0db68373281 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:14.982882 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7989a8b-e7e3-4a76-b030-f0db68373281 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:14.999195 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7989a8b-e7e3-4a76-b030-f0db68373281 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:15.001323 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7989a8b-e7e3-4a76-b030-f0db68373281 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-e7989a8b-e7e3-4a76-b030-f0db68373281 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:15.031067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:15.032157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:15.033040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:15.033040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:15.033040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:15.033040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:15.033040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:15.033040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:15.033040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:15.033040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:15.033040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 389/767] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:15.041794 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7989a8b-e7e3-4a76-b030-f0db68373281 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:15.123408 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 379/768] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:15.125622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce3dce-5929-4d85-a237-1bf3284460d2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:15.143274 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:15.186512 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:15.231925 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7989a8b-e7e3-4a76-b030-f0db68373281 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:15.277933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efff91-2075-49d4-82cf-1ef728a87689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:15.279432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efff91-2075-49d4-82cf-1ef728a87689 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:15.279899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efff91-2075-49d4-82cf-1ef728a87689 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:15.280382 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efff91-2075-49d4-82cf-1ef728a87689 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:15.281297 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efff91-2075-49d4-82cf-1ef728a87689 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:15.281787 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efff91-2075-49d4-82cf-1ef728a87689 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:15.283555 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efff91-2075-49d4-82cf-1ef728a87689 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:15.299288 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 390/769] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:15.319589 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efff91-2075-49d4-82cf-1ef728a87689 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:15.417495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce3dce-5929-4d85-a237-1bf3284460d2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:15.469363 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efff91-2075-49d4-82cf-1ef728a87689 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:15.472873 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:15.511355 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2f189-52f2-4bd0-88e2-731d35507b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:15.512345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2f189-52f2-4bd0-88e2-731d35507b68 None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:15.512846 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2f189-52f2-4bd0-88e2-731d35507b68 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:15.513300 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2f189-52f2-4bd0-88e2-731d35507b68 None admin] PATH_INFO: `/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/02260480264b4f8f8a7e5cb647ac5d9e/roles/fc68d072459d4720b9666c93bd535d63` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:15.515971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2f189-52f2-4bd0-88e2-731d35507b68 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fb003a695490faa0077bbbe5e7d77, user_id=02260480264b4f8f8a7e5cb647ac5d9e, role_id=fc68d072459d4720b9666c93bd535d63)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:15.517638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2f189-52f2-4bd0-88e2-731d35507b68 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:15.535831 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 391/770] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:36:15 2022] PUT /identity/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/02260480264b4f8f8a7e5cb647ac5d9e/roles/fc68d072459d4720b9666c93bd535d63 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:36:15.579332 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e9d57-dad1-415c-bba8-d99986387a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:15.580408 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e9d57-dad1-415c-bba8-d99986387a15 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:15.580840 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e9d57-dad1-415c-bba8-d99986387a15 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:15.581307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e9d57-dad1-415c-bba8-d99986387a15 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:15.582379 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e9d57-dad1-415c-bba8-d99986387a15 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:15.584809 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e9d57-dad1-415c-bba8-d99986387a15 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-5e1e9d57-dad1-415c-bba8-d99986387a15 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:15.611081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:15.612169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 12:36:15.612994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:15.619327 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 380/771] 10.222.0.33 () {56 vars in 1129 bytes} [Tue Oct 4 12:36:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 495 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:15.625211 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2f189-52f2-4bd0-88e2-731d35507b68 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:15.664661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e9d57-dad1-415c-bba8-d99986387a15 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:15.778985 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2f189-52f2-4bd0-88e2-731d35507b68 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:15.831886 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d145a-4fcd-4caa-bd98-16215ed85de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:15.832832 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d145a-4fcd-4caa-bd98-16215ed85de2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:15.833270 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d145a-4fcd-4caa-bd98-16215ed85de2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:15.833690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d145a-4fcd-4caa-bd98-16215ed85de2 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:15.834889 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d145a-4fcd-4caa-bd98-16215ed85de2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:15.835416 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d145a-4fcd-4caa-bd98-16215ed85de2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:15.837211 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d145a-4fcd-4caa-bd98-16215ed85de2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:15.844092 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 392/772] 10.222.0.33 () {56 vars in 1124 bytes} [Tue Oct 4 12:36:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:15.850419 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d145a-4fcd-4caa-bd98-16215ed85de2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:15.885998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:15.952481 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:15.968670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e9d57-dad1-415c-bba8-d99986387a15 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:16.026069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d145a-4fcd-4caa-bd98-16215ed85de2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:16.071627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3bbca-d5a7-41fa-8918-d7ff43f9dcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:16.072193 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3bbca-d5a7-41fa-8918-d7ff43f9dcb0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:16.072475 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3bbca-d5a7-41fa-8918-d7ff43f9dcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:16.072762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3bbca-d5a7-41fa-8918-d7ff43f9dcb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:16.092643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b9074-7505-4167-a2ba-f7609e58e3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:16.094075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b9074-7505-4167-a2ba-f7609e58e3d2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:16.094793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b9074-7505-4167-a2ba-f7609e58e3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:16.095497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b9074-7505-4167-a2ba-f7609e58e3d2 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:16.097102 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b9074-7505-4167-a2ba-f7609e58e3d2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:16.097919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b9074-7505-4167-a2ba-f7609e58e3d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:16.101355 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b9074-7505-4167-a2ba-f7609e58e3d2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:16.107321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3bbca-d5a7-41fa-8918-d7ff43f9dcb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a1f3bbca-d5a7-41fa-8918-d7ff43f9dcb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:16.112825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:16.114861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:16.116403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:16.116403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:16.116403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:16.116403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:16.116403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:16.116403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:16.116403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:16.116403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:16.116403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:16.116403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:16.116403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:36:16.116403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:16.118263 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 393/773] 10.222.0.33 () {56 vars in 1124 bytes} [Tue Oct 4 12:36:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:16.120691 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b9074-7505-4167-a2ba-f7609e58e3d2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:16.122481 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 381/774] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:36:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:16.135110 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3bbca-d5a7-41fa-8918-d7ff43f9dcb0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:16.328523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:16.347060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b9074-7505-4167-a2ba-f7609e58e3d2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:16.414201 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3bbca-d5a7-41fa-8918-d7ff43f9dcb0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:16.418147 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8fd38-7ed3-46e0-a386-9395909b8471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:16.418775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8fd38-7ed3-46e0-a386-9395909b8471 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:16.419054 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8fd38-7ed3-46e0-a386-9395909b8471 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:16.419366 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8fd38-7ed3-46e0-a386-9395909b8471 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:16.420033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8fd38-7ed3-46e0-a386-9395909b8471 None admin] RBAC: Adding query filter params (interface=public, service_id=3d8981a2bdbf4b62a58809e7181794de, region_id=RegionOne) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:16.420344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8fd38-7ed3-46e0-a386-9395909b8471 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:16.421740 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8fd38-7ed3-46e0-a386-9395909b8471 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:16.439570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 394/775] 10.222.0.33 () {56 vars in 1251 bytes} [Tue Oct 4 12:36:16 2022] GET /identity/v3/endpoints?service_id=3d8981a2bdbf4b62a58809e7181794de&interface=public®ion_id=RegionOne => generated 194 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:16.445910 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8fd38-7ed3-46e0-a386-9395909b8471 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:16.502433 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c74ec5-7bf5-49f9-89b7-1432cf454a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:16.503128 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c74ec5-7bf5-49f9-89b7-1432cf454a42 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:16.503411 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c74ec5-7bf5-49f9-89b7-1432cf454a42 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:16.503677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c74ec5-7bf5-49f9-89b7-1432cf454a42 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:16.504376 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c74ec5-7bf5-49f9-89b7-1432cf454a42 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:16.504682 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c74ec5-7bf5-49f9-89b7-1432cf454a42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:16.506116 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c74ec5-7bf5-49f9-89b7-1432cf454a42 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:16.532335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 382/776] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:36:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:16.539067 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c74ec5-7bf5-49f9-89b7-1432cf454a42 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:16.596784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:16.600368 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8fd38-7ed3-46e0-a386-9395909b8471 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:16.643085 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949a129-0455-4108-be29-9069f2613179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:16.643715 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949a129-0455-4108-be29-9069f2613179 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:16.650144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949a129-0455-4108-be29-9069f2613179 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:16.650144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949a129-0455-4108-be29-9069f2613179 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:16.650144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949a129-0455-4108-be29-9069f2613179 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:16.650144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949a129-0455-4108-be29-9069f2613179 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:16.653695 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 395/777] 10.222.0.33 () {56 vars in 1087 bytes} [Tue Oct 4 12:36:16 2022] GET /identity/v3/services => generated 772 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:16.742057 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c74ec5-7bf5-49f9-89b7-1432cf454a42 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:16.784564 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689cba3-c4f8-44b4-8a28-878f2ac1dd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:16.785223 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689cba3-c4f8-44b4-8a28-878f2ac1dd66 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:16.785637 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689cba3-c4f8-44b4-8a28-878f2ac1dd66 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:16.785902 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689cba3-c4f8-44b4-8a28-878f2ac1dd66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:16.800185 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689cba3-c4f8-44b4-8a28-878f2ac1dd66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:16.801682 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689cba3-c4f8-44b4-8a28-878f2ac1dd66 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-7689cba3-c4f8-44b4-8a28-878f2ac1dd66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:16.819255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:16.823532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:16.831633 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:16.831633 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:16.831633 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:16.831633 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 383/778] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:16.832491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949a129-0455-4108-be29-9069f2613179 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:16.869353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:16.991556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949a129-0455-4108-be29-9069f2613179 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:17.031477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f7c29-29dc-4d3a-b370-2aa2c7c5ecbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:17.032096 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f7c29-29dc-4d3a-b370-2aa2c7c5ecbe None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:17.032437 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f7c29-29dc-4d3a-b370-2aa2c7c5ecbe None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:17.032720 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f7c29-29dc-4d3a-b370-2aa2c7c5ecbe None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:17.033371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f7c29-29dc-4d3a-b370-2aa2c7c5ecbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:17.033744 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f7c29-29dc-4d3a-b370-2aa2c7c5ecbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:17.035232 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f7c29-29dc-4d3a-b370-2aa2c7c5ecbe None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:17.048103 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 396/779] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:17.053299 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689cba3-c4f8-44b4-8a28-878f2ac1dd66 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:17.163511 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:17.204663 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689cba3-c4f8-44b4-8a28-878f2ac1dd66 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:17.244417 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad80b9-0304-4218-b3c3-5ee3e74f14c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:17.245077 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad80b9-0304-4218-b3c3-5ee3e74f14c1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:17.245389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad80b9-0304-4218-b3c3-5ee3e74f14c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:17.245633 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad80b9-0304-4218-b3c3-5ee3e74f14c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:17.260783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad80b9-0304-4218-b3c3-5ee3e74f14c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:17.260783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad80b9-0304-4218-b3c3-5ee3e74f14c1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c0ad80b9-0304-4218-b3c3-5ee3e74f14c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:17.278874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:17.282445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:17.292939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:17.292939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:17.292939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:17.292939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:17.292939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:17.292939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:17.292939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:17.292939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:17.292939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 12:36:17.292939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:17.292939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 384/780] 10.222.0.33 () {56 vars in 1117 bytes} [Tue Oct 4 12:36:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:17.298461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f7c29-29dc-4d3a-b370-2aa2c7c5ecbe None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:17.406894 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bd45e-329b-426b-9e5b-424ab43ce1a0 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:17.407255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bd45e-329b-426b-9e5b-424ab43ce1a0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:17.407417 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bd45e-329b-426b-9e5b-424ab43ce1a0 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:17.408806 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 385/781] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:17.416197 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dc714-6024-442e-bb48-c387ff32e3be None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:17.416479 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dc714-6024-442e-bb48-c387ff32e3be None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:17.417032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dc714-6024-442e-bb48-c387ff32e3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:17.486253 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-cf6dc714-6024-442e-bb48-c387ff32e3be None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:17.487499 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cf6dc714-6024-442e-bb48-c387ff32e3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:17.525011 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:17.576330 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f7c29-29dc-4d3a-b370-2aa2c7c5ecbe None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:17.656201 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 386/782] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:17.669077 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbaf55-d6d9-4026-a89d-a8cb4e7724a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:17.669619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbaf55-d6d9-4026-a89d-a8cb4e7724a2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:17.669891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbaf55-d6d9-4026-a89d-a8cb4e7724a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:17.670177 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbaf55-d6d9-4026-a89d-a8cb4e7724a2 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:17.670940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbaf55-d6d9-4026-a89d-a8cb4e7724a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:17.671249 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbaf55-d6d9-4026-a89d-a8cb4e7724a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:17.672850 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbaf55-d6d9-4026-a89d-a8cb4e7724a2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:17.697951 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 397/783] 10.222.0.33 () {56 vars in 1149 bytes} [Tue Oct 4 12:36:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:17.704140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dc714-6024-442e-bb48-c387ff32e3be None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:17.785362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:17.833795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dc714-6024-442e-bb48-c387ff32e3be None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:17.882870 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd1fd2-70b4-472e-8654-da0a769bdfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:17.883550 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd1fd2-70b4-472e-8654-da0a769bdfa6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:17.883835 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd1fd2-70b4-472e-8654-da0a769bdfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:17.884177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd1fd2-70b4-472e-8654-da0a769bdfa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:17.896194 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd1fd2-70b4-472e-8654-da0a769bdfa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:17.897730 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fd1fd2-70b4-472e-8654-da0a769bdfa6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-96fd1fd2-70b4-472e-8654-da0a769bdfa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:17.917361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:17.921221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:17.927225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:17.927225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:17.927225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:17.927225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 387/784] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:17.928066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbaf55-d6d9-4026-a89d-a8cb4e7724a2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:18.094840 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbaf55-d6d9-4026-a89d-a8cb4e7724a2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:18.145458 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd1fd2-70b4-472e-8654-da0a769bdfa6 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:18.148849 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef637ab4-6758-4342-8126-670c3cd46df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:18.149471 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef637ab4-6758-4342-8126-670c3cd46df2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:18.150350 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef637ab4-6758-4342-8126-670c3cd46df2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:18.150620 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef637ab4-6758-4342-8126-670c3cd46df2 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:18.151280 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef637ab4-6758-4342-8126-670c3cd46df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:18.151584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef637ab4-6758-4342-8126-670c3cd46df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:18.153020 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef637ab4-6758-4342-8126-670c3cd46df2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:18.173853 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:18.173853 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 398/785] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:18.179893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef637ab4-6758-4342-8126-670c3cd46df2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:18.304477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef637ab4-6758-4342-8126-670c3cd46df2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:18.362084 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd1fd2-70b4-472e-8654-da0a769bdfa6 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:18.366060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d0825-c34e-41de-8fbd-0178da8b3eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:18.366988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d0825-c34e-41de-8fbd-0178da8b3eb1 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:18.367423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d0825-c34e-41de-8fbd-0178da8b3eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:18.367857 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d0825-c34e-41de-8fbd-0178da8b3eb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:18.376264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d0825-c34e-41de-8fbd-0178da8b3eb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:18.378117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:18.379518 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d0825-c34e-41de-8fbd-0178da8b3eb1 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-7d5d0825-c34e-41de-8fbd-0178da8b3eb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:18.394549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:18.395621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:18.396523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:18.396523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:18.396523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:18.396523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:18.396523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:18.396523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:18.396523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:18.396523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:18.398700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 399/786] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:18.405212 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d0825-c34e-41de-8fbd-0178da8b3eb1 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:18.448807 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31231a81-b71b-4dc6-9c78-e1d18f414443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:18.449397 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31231a81-b71b-4dc6-9c78-e1d18f414443 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:18.449678 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31231a81-b71b-4dc6-9c78-e1d18f414443 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:18.449958 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31231a81-b71b-4dc6-9c78-e1d18f414443 None admin] PATH_INFO: `/v3/services/network` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:18.450657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31231a81-b71b-4dc6-9c78-e1d18f414443 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:18.452232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31231a81-b71b-4dc6-9c78-e1d18f414443 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-31231a81-b71b-4dc6-9c78-e1d18f414443 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:18.470838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:18.472046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 12:36:18.472973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:18.478845 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 388/787] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:18 2022] GET /identity/v3/services/network => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:18.483630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31231a81-b71b-4dc6-9c78-e1d18f414443 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:18.486579 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:18.541409 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d0825-c34e-41de-8fbd-0178da8b3eb1 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:18.588598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f7b16-5a46-4836-9b5a-d53cbcaa3d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:18.589223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f7b16-5a46-4836-9b5a-d53cbcaa3d67 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:18.589516 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f7b16-5a46-4836-9b5a-d53cbcaa3d67 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:18.589791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f7b16-5a46-4836-9b5a-d53cbcaa3d67 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:18.590532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f7b16-5a46-4836-9b5a-d53cbcaa3d67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:18.591510 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f7b16-5a46-4836-9b5a-d53cbcaa3d67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:18.593316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f7b16-5a46-4836-9b5a-d53cbcaa3d67 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:18.605257 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31231a81-b71b-4dc6-9c78-e1d18f414443 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:18.615574 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 400/788] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:18.616875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7a66f-f735-4cae-b18e-469e4cb7a214 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:18.617170 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7a66f-f735-4cae-b18e-469e4cb7a214 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:18.617448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7a66f-f735-4cae-b18e-469e4cb7a214 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:18.619589 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 401/789] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:18.620416 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e05090-10ba-4bcd-8139-06b96f742e2a None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:18.620698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e05090-10ba-4bcd-8139-06b96f742e2a None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:18.620972 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e05090-10ba-4bcd-8139-06b96f742e2a None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:18.622572 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 402/790] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:18.623013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e05090-10ba-4bcd-8139-06b96f742e2a None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:18.650332 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e7032-bba2-47f6-8365-bd5fba8c3c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:18.651318 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e7032-bba2-47f6-8365-bd5fba8c3c8d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:18.651600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e7032-bba2-47f6-8365-bd5fba8c3c8d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:18.652023 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e7032-bba2-47f6-8365-bd5fba8c3c8d None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:18.652652 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e7032-bba2-47f6-8365-bd5fba8c3c8d None admin] RBAC: Adding query filter params (name=network) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:18.652947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e7032-bba2-47f6-8365-bd5fba8c3c8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:18.654360 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e7032-bba2-47f6-8365-bd5fba8c3c8d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:18.659746 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 389/791] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:18.660788 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2c63c-7058-4c0e-bdd3-564ccd6f7cce None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:18.661063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2c63c-7058-4c0e-bdd3-564ccd6f7cce None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:18.661326 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2c63c-7058-4c0e-bdd3-564ccd6f7cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:18.684892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1cc2c63c-7058-4c0e-bdd3-564ccd6f7cce None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:18.686151 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1cc2c63c-7058-4c0e-bdd3-564ccd6f7cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:18.722420 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:18.755400 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e05090-10ba-4bcd-8139-06b96f742e2a None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:18.786074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 390/792] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:18.786074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16487fec-09eb-46cd-98db-8dead692b016 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:18.786074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16487fec-09eb-46cd-98db-8dead692b016 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:18.786074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16487fec-09eb-46cd-98db-8dead692b016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:18.792127 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:18.812065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-16487fec-09eb-46cd-98db-8dead692b016 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:18.814287 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-16487fec-09eb-46cd-98db-8dead692b016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:18.820764 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cdea7-cf20-4ac6-9868-7ff1b319595b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:18.821835 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cdea7-cf20-4ac6-9868-7ff1b319595b None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:18.822323 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cdea7-cf20-4ac6-9868-7ff1b319595b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:18.822782 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cdea7-cf20-4ac6-9868-7ff1b319595b None admin] PATH_INFO: `/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/58d82737c2864faf8fef3b53167c4dfc/roles/943fc0edf23046a69ba6102f2c1e544c` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:18.847944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cdea7-cf20-4ac6-9868-7ff1b319595b None admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fb003a695490faa0077bbbe5e7d77, user_id=58d82737c2864faf8fef3b53167c4dfc, role_id=943fc0edf23046a69ba6102f2c1e544c)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:18.850482 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336cdea7-cf20-4ac6-9868-7ff1b319595b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:18.875981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 403/793] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:36:18 2022] PUT /identity/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/58d82737c2864faf8fef3b53167c4dfc/roles/943fc0edf23046a69ba6102f2c1e544c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:36:18.877742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cdea7-cf20-4ac6-9868-7ff1b319595b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:18.948185 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 391/794] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:19.086886 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cdea7-cf20-4ac6-9868-7ff1b319595b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:19.179826 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5d7af-2bbe-4978-9d2c-eb5ceb2f8b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:19.180938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5d7af-2bbe-4978-9d2c-eb5ceb2f8b9d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:19.181429 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5d7af-2bbe-4978-9d2c-eb5ceb2f8b9d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:19.181909 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5d7af-2bbe-4978-9d2c-eb5ceb2f8b9d None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:19.183181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5d7af-2bbe-4978-9d2c-eb5ceb2f8b9d None admin] RBAC: Adding query filter params (type=network) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:19.183721 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5d7af-2bbe-4978-9d2c-eb5ceb2f8b9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:19.186241 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5d7af-2bbe-4978-9d2c-eb5ceb2f8b9d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:19.198347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 404/795] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:19.206654 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16487fec-09eb-46cd-98db-8dead692b016 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:19.267907 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5d7af-2bbe-4978-9d2c-eb5ceb2f8b9d None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:19.367503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:19.421572 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:19.429341 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16487fec-09eb-46cd-98db-8dead692b016 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:19.499804 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81313c-b9c3-4082-9438-212e413e03a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:19.501205 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81313c-b9c3-4082-9438-212e413e03a0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:19.501874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81313c-b9c3-4082-9438-212e413e03a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:19.502541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81313c-b9c3-4082-9438-212e413e03a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:19.504099 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81313c-b9c3-4082-9438-212e413e03a0 None admin] RBAC: Adding query filter params (interface=public, service_id=9cab9f60a69f4fcbb59d4863adf611d7, region_id=RegionOne) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:19.504880 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81313c-b9c3-4082-9438-212e413e03a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:19.507639 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81313c-b9c3-4082-9438-212e413e03a0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:19.530210 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 392/796] 10.222.0.33 () {56 vars in 1251 bytes} [Tue Oct 4 12:36:19 2022] GET /identity/v3/endpoints?service_id=9cab9f60a69f4fcbb59d4863adf611d7&interface=public®ion_id=RegionOne => generated 194 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:19.532116 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81313c-b9c3-4082-9438-212e413e03a0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:19.556833 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5d7af-2bbe-4978-9d2c-eb5ceb2f8b9d None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:19.613370 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19971084-7523-43b5-911e-5af471655cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:19.614256 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19971084-7523-43b5-911e-5af471655cd2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:19.614727 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19971084-7523-43b5-911e-5af471655cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:19.615141 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19971084-7523-43b5-911e-5af471655cd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:19.630714 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19971084-7523-43b5-911e-5af471655cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-19971084-7523-43b5-911e-5af471655cd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:19.638439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:19.640491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 405/797] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:36:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:19.643155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19971084-7523-43b5-911e-5af471655cd2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:19.689706 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81313c-b9c3-4082-9438-212e413e03a0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:19.731500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839b570-be48-4460-9d39-bc944537295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:19.732231 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839b570-be48-4460-9d39-bc944537295c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:19.732525 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839b570-be48-4460-9d39-bc944537295c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:19.732805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839b570-be48-4460-9d39-bc944537295c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:19.745611 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839b570-be48-4460-9d39-bc944537295c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c839b570-be48-4460-9d39-bc944537295c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:19.751212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:19.752911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 393/798] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:36:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:19.755297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e5f6b-d675-4254-b970-766e230ea3f8 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:19.758406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e5f6b-d675-4254-b970-766e230ea3f8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:19.758406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e5f6b-d675-4254-b970-766e230ea3f8 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:19.758406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 394/799] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:19.758406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e5f6b-d675-4254-b970-766e230ea3f8 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:19.781253 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:19.789413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:19.795301 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19971084-7523-43b5-911e-5af471655cd2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:19.835543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61271871-fca8-430c-8eaf-20d97519c89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:19.836162 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61271871-fca8-430c-8eaf-20d97519c89e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:19.836443 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61271871-fca8-430c-8eaf-20d97519c89e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:19.836720 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61271871-fca8-430c-8eaf-20d97519c89e None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:19.837915 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61271871-fca8-430c-8eaf-20d97519c89e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:19.839362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61271871-fca8-430c-8eaf-20d97519c89e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:19.844826 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 406/800] 10.222.0.33 () {56 vars in 1087 bytes} [Tue Oct 4 12:36:19 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:19.846088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d85f8-480e-4f9f-941f-133bf6a8cbf1 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:19.846393 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d85f8-480e-4f9f-941f-133bf6a8cbf1 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:19.846660 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d85f8-480e-4f9f-941f-133bf6a8cbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:19.871698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-a87d85f8-480e-4f9f-941f-133bf6a8cbf1 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:19.873047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-a87d85f8-480e-4f9f-941f-133bf6a8cbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:19.911381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e5f6b-d675-4254-b970-766e230ea3f8 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:19.970687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc75a5b-70d5-4424-84e4-6e812a44f788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:19.972016 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc75a5b-70d5-4424-84e4-6e812a44f788 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:19.972478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc75a5b-70d5-4424-84e4-6e812a44f788 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:19.972983 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc75a5b-70d5-4424-84e4-6e812a44f788 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:19.975333 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc75a5b-70d5-4424-84e4-6e812a44f788 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:19.975883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc75a5b-70d5-4424-84e4-6e812a44f788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:19.978943 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc75a5b-70d5-4424-84e4-6e812a44f788 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:19.993951 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 407/801] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:19.996136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d85f8-480e-4f9f-941f-133bf6a8cbf1 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:20.002306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 395/802] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:36:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:20.009909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc75a5b-70d5-4424-84e4-6e812a44f788 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:20.163321 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:20.185389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d85f8-480e-4f9f-941f-133bf6a8cbf1 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:20.259766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4964e2-d6f8-4561-9b48-730a2fa6ac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:20.260667 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4964e2-d6f8-4561-9b48-730a2fa6ac5a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:20.261146 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4964e2-d6f8-4561-9b48-730a2fa6ac5a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:20.261607 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4964e2-d6f8-4561-9b48-730a2fa6ac5a None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:20.266112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4964e2-d6f8-4561-9b48-730a2fa6ac5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:20.266112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4964e2-d6f8-4561-9b48-730a2fa6ac5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:20.266112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4964e2-d6f8-4561-9b48-730a2fa6ac5a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:20.283075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 408/803] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:36:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:20.291828 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4964e2-d6f8-4561-9b48-730a2fa6ac5a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:20.394513 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:20.413657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc75a5b-70d5-4424-84e4-6e812a44f788 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:20.453578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe8f14-fa40-4ce9-8055-7b70f1e3d5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:20.454584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe8f14-fa40-4ce9-8055-7b70f1e3d5cc None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:20.454584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe8f14-fa40-4ce9-8055-7b70f1e3d5cc None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:20.454874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe8f14-fa40-4ce9-8055-7b70f1e3d5cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:20.467541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe8f14-fa40-4ce9-8055-7b70f1e3d5cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:20.469010 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe8f14-fa40-4ce9-8055-7b70f1e3d5cc None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-f4fe8f14-fa40-4ce9-8055-7b70f1e3d5cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:20.486650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:20.488551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:20.491691 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:20.491691 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:20.491691 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:20.491691 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 396/804] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 481 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:20.505104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe8f14-fa40-4ce9-8055-7b70f1e3d5cc None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:20.591169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4964e2-d6f8-4561-9b48-730a2fa6ac5a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:20.657614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a141c5-3c96-4852-99ae-081f9a528ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:20.658298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a141c5-3c96-4852-99ae-081f9a528ad2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:20.658566 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a141c5-3c96-4852-99ae-081f9a528ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:20.658830 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a141c5-3c96-4852-99ae-081f9a528ad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:20.670938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a141c5-3c96-4852-99ae-081f9a528ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:20.672331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a141c5-3c96-4852-99ae-081f9a528ad2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:20.681795 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-13a141c5-3c96-4852-99ae-081f9a528ad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:20.689052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:20.690977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:20.693565 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:20.693565 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:20.693565 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:20.693565 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 409/805] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:20.693565 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a141c5-3c96-4852-99ae-081f9a528ad2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:20.700825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe8f14-fa40-4ce9-8055-7b70f1e3d5cc None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:20.764623 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6d28e-6664-4a3b-9eb4-a6dfae480745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:20.766177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6d28e-6664-4a3b-9eb4-a6dfae480745 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:20.774672 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6d28e-6664-4a3b-9eb4-a6dfae480745 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:20.775677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6d28e-6664-4a3b-9eb4-a6dfae480745 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:20.777082 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6d28e-6664-4a3b-9eb4-a6dfae480745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:20.777926 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6d28e-6664-4a3b-9eb4-a6dfae480745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:20.786957 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6d28e-6664-4a3b-9eb4-a6dfae480745 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:20.790913 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:20.803086 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 397/806] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:20.804273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6d28e-6664-4a3b-9eb4-a6dfae480745 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:20.808857 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a141c5-3c96-4852-99ae-081f9a528ad2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:20.847951 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cca5df-aca9-4e07-981a-2884b0215426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:20.848629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cca5df-aca9-4e07-981a-2884b0215426 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:20.848909 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cca5df-aca9-4e07-981a-2884b0215426 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:20.849181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cca5df-aca9-4e07-981a-2884b0215426 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:20.849772 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cca5df-aca9-4e07-981a-2884b0215426 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:20.851347 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cca5df-aca9-4e07-981a-2884b0215426 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-82cca5df-aca9-4e07-981a-2884b0215426 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:20.862260 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:20.863590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:20.864626 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 410/807] 10.222.0.33 () {56 vars in 1129 bytes} [Tue Oct 4 12:36:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:20.865448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cca5df-aca9-4e07-981a-2884b0215426 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:20.934409 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6d28e-6664-4a3b-9eb4-a6dfae480745 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:20.962790 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:20.978269 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069b03c-ef85-47ab-9acd-4715d9582926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:20.978951 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069b03c-ef85-47ab-9acd-4715d9582926 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:20.979233 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069b03c-ef85-47ab-9acd-4715d9582926 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:20.979503 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069b03c-ef85-47ab-9acd-4715d9582926 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:20.980152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069b03c-ef85-47ab-9acd-4715d9582926 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:20.980453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069b03c-ef85-47ab-9acd-4715d9582926 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:20.981830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069b03c-ef85-47ab-9acd-4715d9582926 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:20.997601 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 398/808] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:20.998713 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069b03c-ef85-47ab-9acd-4715d9582926 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:21.017016 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cca5df-aca9-4e07-981a-2884b0215426 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:21.070107 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3876b3-d0f5-47a0-91ba-863f0e021ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:21.071493 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3876b3-d0f5-47a0-91ba-863f0e021ff2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:21.071957 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3876b3-d0f5-47a0-91ba-863f0e021ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:21.072582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3876b3-d0f5-47a0-91ba-863f0e021ff2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:21.073234 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:21.083888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3876b3-d0f5-47a0-91ba-863f0e021ff2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:21.085849 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3876b3-d0f5-47a0-91ba-863f0e021ff2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-aa3876b3-d0f5-47a0-91ba-863f0e021ff2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:21.105842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:21.107231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:21.108404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:21.108404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:21.108404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:21.108404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:21.108404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:21.108404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:21.108404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:21.108404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:21.108404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 12:36:21.108404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:21.108404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 411/809] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:36:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:21.109101 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3876b3-d0f5-47a0-91ba-863f0e021ff2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:21.139250 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069b03c-ef85-47ab-9acd-4715d9582926 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:21.191439 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb789aa6-6091-4e44-8cb0-a1d8b973ca80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:21.192280 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb789aa6-6091-4e44-8cb0-a1d8b973ca80 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:21.192713 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb789aa6-6091-4e44-8cb0-a1d8b973ca80 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:21.193269 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb789aa6-6091-4e44-8cb0-a1d8b973ca80 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:21.193976 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb789aa6-6091-4e44-8cb0-a1d8b973ca80 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:21.194306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb789aa6-6091-4e44-8cb0-a1d8b973ca80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:21.196069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb789aa6-6091-4e44-8cb0-a1d8b973ca80 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:21.207753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 399/810] 10.222.0.33 () {56 vars in 1124 bytes} [Tue Oct 4 12:36:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:21.208877 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb789aa6-6091-4e44-8cb0-a1d8b973ca80 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:21.260797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3876b3-d0f5-47a0-91ba-863f0e021ff2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:21.293909 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:21.309850 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75d766-e463-4ad1-8223-77dd06bad5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:21.311142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75d766-e463-4ad1-8223-77dd06bad5cf None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:21.311577 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75d766-e463-4ad1-8223-77dd06bad5cf None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:21.311998 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75d766-e463-4ad1-8223-77dd06bad5cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:21.324139 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75d766-e463-4ad1-8223-77dd06bad5cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:21.326453 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75d766-e463-4ad1-8223-77dd06bad5cf None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-7d75d766-e463-4ad1-8223-77dd06bad5cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:21.343615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:21.344856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:21.346057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:21.346057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:21.346057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:21.346057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:21.346057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:21.346057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:21.346057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:21.346057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:21.346057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:36:21.346057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:21.349296 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 412/811] 10.222.0.33 () {56 vars in 1093 bytes} [Tue Oct 4 12:36:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:21.350653 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75d766-e463-4ad1-8223-77dd06bad5cf None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:21.354662 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb789aa6-6091-4e44-8cb0-a1d8b973ca80 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:21.385217 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:21.408748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294961b6-544a-44b9-ac7f-a420b3e1029d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:21.409387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294961b6-544a-44b9-ac7f-a420b3e1029d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:21.409667 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294961b6-544a-44b9-ac7f-a420b3e1029d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:21.409944 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294961b6-544a-44b9-ac7f-a420b3e1029d None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:21.410781 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294961b6-544a-44b9-ac7f-a420b3e1029d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:21.411096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294961b6-544a-44b9-ac7f-a420b3e1029d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:21.412564 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294961b6-544a-44b9-ac7f-a420b3e1029d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:21.434815 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 400/812] 10.222.0.33 () {56 vars in 1135 bytes} [Tue Oct 4 12:36:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:21.435932 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294961b6-544a-44b9-ac7f-a420b3e1029d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:21.477032 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75d766-e463-4ad1-8223-77dd06bad5cf None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:21.516916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a7a77-6d7e-4ae8-a0f0-ebb980d739b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:21.517992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a7a77-6d7e-4ae8-a0f0-ebb980d739b9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:21.518500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a7a77-6d7e-4ae8-a0f0-ebb980d739b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:21.519217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a7a77-6d7e-4ae8-a0f0-ebb980d739b9 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:21.520044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a7a77-6d7e-4ae8-a0f0-ebb980d739b9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:21.520557 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a7a77-6d7e-4ae8-a0f0-ebb980d739b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:21.522373 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a7a77-6d7e-4ae8-a0f0-ebb980d739b9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:21.529836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 413/813] 10.222.0.33 () {56 vars in 1124 bytes} [Tue Oct 4 12:36:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:21.531125 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a7a77-6d7e-4ae8-a0f0-ebb980d739b9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:21.562640 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294961b6-544a-44b9-ac7f-a420b3e1029d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:21.599867 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:21.609179 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32508b-d417-463a-86f9-255d0c2f2dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:21.610677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32508b-d417-463a-86f9-255d0c2f2dc0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:21.611367 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32508b-d417-463a-86f9-255d0c2f2dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:21.611789 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32508b-d417-463a-86f9-255d0c2f2dc0 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:21.612887 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32508b-d417-463a-86f9-255d0c2f2dc0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:21.613433 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32508b-d417-463a-86f9-255d0c2f2dc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:21.615143 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32508b-d417-463a-86f9-255d0c2f2dc0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:21.639059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 401/814] 10.222.0.33 () {56 vars in 1133 bytes} [Tue Oct 4 12:36:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:21.640372 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32508b-d417-463a-86f9-255d0c2f2dc0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:21.667065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a7a77-6d7e-4ae8-a0f0-ebb980d739b9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:21.684991 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:21.715900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed25e02-201c-47dc-9ced-32b0c0ea69be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:21.716801 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed25e02-201c-47dc-9ced-32b0c0ea69be None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:21.717482 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed25e02-201c-47dc-9ced-32b0c0ea69be None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:21.717840 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed25e02-201c-47dc-9ced-32b0c0ea69be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:21.733055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed25e02-201c-47dc-9ced-32b0c0ea69be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:21.734642 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed25e02-201c-47dc-9ced-32b0c0ea69be None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-0ed25e02-201c-47dc-9ced-32b0c0ea69be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:21.753501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:21.754588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:21.755748 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:21.755748 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:21.755748 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:21.755748 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 414/815] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:21.756423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed25e02-201c-47dc-9ced-32b0c0ea69be None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:21.775103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32508b-d417-463a-86f9-255d0c2f2dc0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:21.820419 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443c711-e69f-4ca3-add5-8209ce470e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:21.821492 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443c711-e69f-4ca3-add5-8209ce470e77 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:21.821956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443c711-e69f-4ca3-add5-8209ce470e77 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:21.822438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443c711-e69f-4ca3-add5-8209ce470e77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:21.823660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443c711-e69f-4ca3-add5-8209ce470e77 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:21.825725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443c711-e69f-4ca3-add5-8209ce470e77 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:21.862834 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 402/816] 10.222.0.33 () {60 vars in 1144 bytes} [Tue Oct 4 12:36:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:36:21.866687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443c711-e69f-4ca3-add5-8209ce470e77 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:21.921545 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:21.927072 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed25e02-201c-47dc-9ced-32b0c0ea69be None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:22.010760 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea65e5-39f9-4701-ab10-efde2191ac71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:22.011401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea65e5-39f9-4701-ab10-efde2191ac71 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:22.011681 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea65e5-39f9-4701-ab10-efde2191ac71 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:22.011954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea65e5-39f9-4701-ab10-efde2191ac71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:22.081635 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:22.102774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443c711-e69f-4ca3-add5-8209ce470e77 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:22.166663 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98d395-f250-44f4-8896-3fcd06f22329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:22.167717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98d395-f250-44f4-8896-3fcd06f22329 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:22.168214 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98d395-f250-44f4-8896-3fcd06f22329 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:22.168702 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98d395-f250-44f4-8896-3fcd06f22329 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:22.169901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98d395-f250-44f4-8896-3fcd06f22329 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:22.173572 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea65e5-39f9-4701-ab10-efde2191ac71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:22.174250 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98d395-f250-44f4-8896-3fcd06f22329 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:22.176222 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea65e5-39f9-4701-ab10-efde2191ac71 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:22.176874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98d395-f250-44f4-8896-3fcd06f22329 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:22.203241 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 403/817] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:22.213810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98d395-f250-44f4-8896-3fcd06f22329 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:22.384396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:22.398059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98d395-f250-44f4-8896-3fcd06f22329 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:22.400559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 415/818] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:36:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:36:22.403512 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313cc20-424e-4cc2-bd52-18f633628c0b None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:22.403861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313cc20-424e-4cc2-bd52-18f633628c0b None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:22.404076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313cc20-424e-4cc2-bd52-18f633628c0b None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:22.408307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 416/819] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:22.411164 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313cc20-424e-4cc2-bd52-18f633628c0b None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:22.451145 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4860f0f-cb8f-47c4-a248-b262ecc96920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:22.451916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4860f0f-cb8f-47c4-a248-b262ecc96920 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:22.452220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4860f0f-cb8f-47c4-a248-b262ecc96920 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:22.452478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4860f0f-cb8f-47c4-a248-b262ecc96920 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:22.461519 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4860f0f-cb8f-47c4-a248-b262ecc96920 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:22.463078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4860f0f-cb8f-47c4-a248-b262ecc96920 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a4860f0f-cb8f-47c4-a248-b262ecc96920 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:22.476167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:22.477131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:22.478112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:22.478112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:22.478112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:22.478112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:22.478112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:22.478112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:22.478112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:22.478112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:22.478112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 404/820] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:22.478974 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7521d28-6417-4f91-8020-ddc35581491f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:22.479259 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7521d28-6417-4f91-8020-ddc35581491f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:22.479541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7521d28-6417-4f91-8020-ddc35581491f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:22.498974 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f7521d28-6417-4f91-8020-ddc35581491f None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:22.500187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f7521d28-6417-4f91-8020-ddc35581491f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:22.508409 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:22.585462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 405/821] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:22.587601 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401942b-7100-4686-badc-20cbbca4829b None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:22.587899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401942b-7100-4686-badc-20cbbca4829b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:22.588147 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401942b-7100-4686-badc-20cbbca4829b None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:22.596038 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 406/822] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:22.596038 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401942b-7100-4686-badc-20cbbca4829b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:22.617970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313cc20-424e-4cc2-bd52-18f633628c0b None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:22.681844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e1bb7-a640-4605-8a5b-b6f2ff45dfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:22.683491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e1bb7-a640-4605-8a5b-b6f2ff45dfac None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:22.684233 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e1bb7-a640-4605-8a5b-b6f2ff45dfac None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:22.685006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e1bb7-a640-4605-8a5b-b6f2ff45dfac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:22.704895 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e1bb7-a640-4605-8a5b-b6f2ff45dfac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:22.708472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e1bb7-a640-4605-8a5b-b6f2ff45dfac None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:22.708691 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401942b-7100-4686-badc-20cbbca4829b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-aa8e1bb7-a640-4605-8a5b-b6f2ff45dfac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:22.739066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:22.740921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:22.743906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:22.743906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:22.743906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:22.747571 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 417/823] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:22.747911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbba34-3ed2-4ae5-9597-61d2f0531752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:22.748556 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbba34-3ed2-4ae5-9597-61d2f0531752 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:22.748827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbba34-3ed2-4ae5-9597-61d2f0531752 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:22.749070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbba34-3ed2-4ae5-9597-61d2f0531752 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:22.749776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbba34-3ed2-4ae5-9597-61d2f0531752 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:22.750107 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbba34-3ed2-4ae5-9597-61d2f0531752 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:22.751566 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cbba34-3ed2-4ae5-9597-61d2f0531752 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:22.753019 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebc8c8-39de-49e6-8b1c-7f0eaafa0d96 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:22.753846 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebc8c8-39de-49e6-8b1c-7f0eaafa0d96 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:22.754660 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebc8c8-39de-49e6-8b1c-7f0eaafa0d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:22.770608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 407/824] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:22.770608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbba34-3ed2-4ae5-9597-61d2f0531752 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:22.774451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:22.793016 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-ddebc8c8-39de-49e6-8b1c-7f0eaafa0d96 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:22.795484 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-ddebc8c8-39de-49e6-8b1c-7f0eaafa0d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:22.888180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:22.894663 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbba34-3ed2-4ae5-9597-61d2f0531752 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:22.928850 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 418/825] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:22.929807 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebc8c8-39de-49e6-8b1c-7f0eaafa0d96 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:22.937038 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a573c68-c631-4143-98c0-d650bb39fd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:22.937661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a573c68-c631-4143-98c0-d650bb39fd4b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:22.937938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a573c68-c631-4143-98c0-d650bb39fd4b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:22.938251 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a573c68-c631-4143-98c0-d650bb39fd4b None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:22.938900 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a573c68-c631-4143-98c0-d650bb39fd4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:22.939196 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a573c68-c631-4143-98c0-d650bb39fd4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:22.940687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a573c68-c631-4143-98c0-d650bb39fd4b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:22.962504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 408/826] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:22.963965 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a573c68-c631-4143-98c0-d650bb39fd4b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:23.056954 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:23.093050 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebc8c8-39de-49e6-8b1c-7f0eaafa0d96 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:23.107684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a573c68-c631-4143-98c0-d650bb39fd4b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:23.156613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28f5a3-695c-42b9-87be-ef6c07dca729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:23.157625 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28f5a3-695c-42b9-87be-ef6c07dca729 None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:23.158095 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28f5a3-695c-42b9-87be-ef6c07dca729 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:23.158505 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d530308-b0f6-44bc-9bf0-ad11384b687b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:23.158665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28f5a3-695c-42b9-87be-ef6c07dca729 None admin] PATH_INFO: `/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/332fdff1d02a4664923ddbe0478eaa7a/roles/fc68d072459d4720b9666c93bd535d63` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:23.159701 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d530308-b0f6-44bc-9bf0-ad11384b687b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:23.160187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d530308-b0f6-44bc-9bf0-ad11384b687b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:23.160718 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d530308-b0f6-44bc-9bf0-ad11384b687b None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:23.161295 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28f5a3-695c-42b9-87be-ef6c07dca729 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fb003a695490faa0077bbbe5e7d77, user_id=332fdff1d02a4664923ddbe0478eaa7a, role_id=fc68d072459d4720b9666c93bd535d63)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:23.163161 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28f5a3-695c-42b9-87be-ef6c07dca729 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:23.183633 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d530308-b0f6-44bc-9bf0-ad11384b687b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:23.188567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 419/827] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:36:22 2022] PUT /identity/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/332fdff1d02a4664923ddbe0478eaa7a/roles/fc68d072459d4720b9666c93bd535d63 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:36:23.189702 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28f5a3-695c-42b9-87be-ef6c07dca729 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-3d530308-b0f6-44bc-9bf0-ad11384b687b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:23.193001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:23.194248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:23.195218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:23.195218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:23.195218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:23.195218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:23.195218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:23.195218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:23.195218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:23.195218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:23.195218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:23.195218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:23.195218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:36:23.195218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:23.202246 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 409/828] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:36:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:23.205610 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d530308-b0f6-44bc-9bf0-ad11384b687b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:23.253013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:23.382991 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28f5a3-695c-42b9-87be-ef6c07dca729 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:23.392153 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d530308-b0f6-44bc-9bf0-ad11384b687b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:23.435333 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724582ef-390f-4d34-8d9f-1f443a289ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:23.435956 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724582ef-390f-4d34-8d9f-1f443a289ee4 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:23.436192 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724582ef-390f-4d34-8d9f-1f443a289ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:23.436461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724582ef-390f-4d34-8d9f-1f443a289ee4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:23.439377 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:23.448963 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724582ef-390f-4d34-8d9f-1f443a289ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:23.451988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724582ef-390f-4d34-8d9f-1f443a289ee4 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:23.459586 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255c5b0-6b34-4535-bae5-c2a261a4ca4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:23.460436 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255c5b0-6b34-4535-bae5-c2a261a4ca4f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:23.460730 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255c5b0-6b34-4535-bae5-c2a261a4ca4f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:23.461029 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255c5b0-6b34-4535-bae5-c2a261a4ca4f None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:23.461701 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255c5b0-6b34-4535-bae5-c2a261a4ca4f None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:23.462594 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255c5b0-6b34-4535-bae5-c2a261a4ca4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:23.464022 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255c5b0-6b34-4535-bae5-c2a261a4ca4f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-724582ef-390f-4d34-8d9f-1f443a289ee4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:23.472553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:23.473700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:23.474913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:23.474913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:23.474913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:23.474913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:23.474913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:23.474913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:23.474913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:23.474913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:23.474913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 420/829] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:23.475969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724582ef-390f-4d34-8d9f-1f443a289ee4 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:23.484751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 410/830] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:36:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:23.485775 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255c5b0-6b34-4535-bae5-c2a261a4ca4f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:23.646950 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255c5b0-6b34-4535-bae5-c2a261a4ca4f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:23.684608 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:23.693622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724582ef-390f-4d34-8d9f-1f443a289ee4 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:23.723404 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c26c2-9b24-42f5-b783-5d2dd915b796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:23.724802 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c26c2-9b24-42f5-b783-5d2dd915b796 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:23.725462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c26c2-9b24-42f5-b783-5d2dd915b796 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:23.726299 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c26c2-9b24-42f5-b783-5d2dd915b796 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:23.728308 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c26c2-9b24-42f5-b783-5d2dd915b796 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:23.729093 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c26c2-9b24-42f5-b783-5d2dd915b796 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:23.732055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c26c2-9b24-42f5-b783-5d2dd915b796 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:23.766165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 411/831] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:23.766165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c26c2-9b24-42f5-b783-5d2dd915b796 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:23.775465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9f562-1b1d-4e79-aeec-c277774245db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:23.778093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9f562-1b1d-4e79-aeec-c277774245db None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:23.778093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9f562-1b1d-4e79-aeec-c277774245db None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:23.778093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9f562-1b1d-4e79-aeec-c277774245db None admin] PATH_INFO: `/v3/services/network` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:23.778093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9f562-1b1d-4e79-aeec-c277774245db None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:23.778728 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9f562-1b1d-4e79-aeec-c277774245db None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-ead9f562-1b1d-4e79-aeec-c277774245db None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:23.790078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:23.792334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 12:36:23.794025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:23.797715 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 421/832] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:23 2022] GET /identity/v3/services/network => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:23.798511 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9f562-1b1d-4e79-aeec-c277774245db None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:23.871348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:23.954541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c26c2-9b24-42f5-b783-5d2dd915b796 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:23.971328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9f562-1b1d-4e79-aeec-c277774245db None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:23.999823 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b257f8-0df1-4c8c-bae5-d4c57a097322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:24.000693 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b257f8-0df1-4c8c-bae5-d4c57a097322 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:24.001038 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b257f8-0df1-4c8c-bae5-d4c57a097322 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:24.001382 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b257f8-0df1-4c8c-bae5-d4c57a097322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:24.015809 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b257f8-0df1-4c8c-bae5-d4c57a097322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:24.017869 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b257f8-0df1-4c8c-bae5-d4c57a097322 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-b3b257f8-0df1-4c8c-bae5-d4c57a097322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:24.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:24.040699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:24.042757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:24.042757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:24.042757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:24.042757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 412/833] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:24.043261 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b257f8-0df1-4c8c-bae5-d4c57a097322 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:24.047567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d75808-d040-4053-848e-9c7ab444f0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:24.051080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d75808-d040-4053-848e-9c7ab444f0c6 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:24.051848 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d75808-d040-4053-848e-9c7ab444f0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:24.052386 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d75808-d040-4053-848e-9c7ab444f0c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:24.053991 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d75808-d040-4053-848e-9c7ab444f0c6 None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=02260480264b4f8f8a7e5cb647ac5d9e) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:24.054751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d75808-d040-4053-848e-9c7ab444f0c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:24.060161 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d75808-d040-4053-848e-9c7ab444f0c6 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:24.084448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 422/834] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:36:23 2022] GET /identity/v3/role_assignments?user.id=02260480264b4f8f8a7e5cb647ac5d9e&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=fc68d072459d4720b9666c93bd535d63 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:24.084448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d75808-d040-4053-848e-9c7ab444f0c6 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:24.154298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:24.219729 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:24.227232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b257f8-0df1-4c8c-bae5-d4c57a097322 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:24.274882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfde6d-338f-457b-ac02-4f815ebdb6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:24.275695 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfde6d-338f-457b-ac02-4f815ebdb6ea None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:24.276072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfde6d-338f-457b-ac02-4f815ebdb6ea None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:24.276444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfde6d-338f-457b-ac02-4f815ebdb6ea None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:24.277647 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfde6d-338f-457b-ac02-4f815ebdb6ea None admin] RBAC: Adding query filter params (name=network) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:24.277647 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfde6d-338f-457b-ac02-4f815ebdb6ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:24.282074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfde6d-338f-457b-ac02-4f815ebdb6ea None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:24.289850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 413/835] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:24.293131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfde6d-338f-457b-ac02-4f815ebdb6ea None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:24.359835 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d75808-d040-4053-848e-9c7ab444f0c6 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:24.412070 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b30cb-09a4-4f37-b665-d7de643a1573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:24.413140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b30cb-09a4-4f37-b665-d7de643a1573 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:24.413575 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b30cb-09a4-4f37-b665-d7de643a1573 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:24.414003 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b30cb-09a4-4f37-b665-d7de643a1573 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:24.415478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b30cb-09a4-4f37-b665-d7de643a1573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:24.415944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b30cb-09a4-4f37-b665-d7de643a1573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:24.417521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b30cb-09a4-4f37-b665-d7de643a1573 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:24.432066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 423/836] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:24.441238 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b30cb-09a4-4f37-b665-d7de643a1573 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:24.512949 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfde6d-338f-457b-ac02-4f815ebdb6ea None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:24.556367 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d63996-597f-44a8-afc0-da7a95931e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:24.557674 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d63996-597f-44a8-afc0-da7a95931e64 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:24.558198 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d63996-597f-44a8-afc0-da7a95931e64 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:24.558677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d63996-597f-44a8-afc0-da7a95931e64 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:24.560067 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d63996-597f-44a8-afc0-da7a95931e64 None admin] RBAC: Adding query filter params (type=network) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:24.560601 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d63996-597f-44a8-afc0-da7a95931e64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:24.562278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d63996-597f-44a8-afc0-da7a95931e64 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:24.572344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 414/837] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:24.579176 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d63996-597f-44a8-afc0-da7a95931e64 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:24.615716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:24.640951 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:24.661420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b30cb-09a4-4f37-b665-d7de643a1573 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:24.725851 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fa797-6625-4a3d-ab2c-630134f60cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:24.726973 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fa797-6625-4a3d-ab2c-630134f60cbc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:24.727532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fa797-6625-4a3d-ab2c-630134f60cbc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:24.728014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fa797-6625-4a3d-ab2c-630134f60cbc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:24.742549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fa797-6625-4a3d-ab2c-630134f60cbc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:24.745915 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fa797-6625-4a3d-ab2c-630134f60cbc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:24.772175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d63996-597f-44a8-afc0-da7a95931e64 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-310fa797-6625-4a3d-ab2c-630134f60cbc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:24.774213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:24.775172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:24.776110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:24.776110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:24.776110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:24.776110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:24.776110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:24.776110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:24.776110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:24.776110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:24.776110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 12:36:24.776110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:24.776634 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 424/838] 10.222.0.33 () {56 vars in 1117 bytes} [Tue Oct 4 12:36:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:24.785783 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fa797-6625-4a3d-ab2c-630134f60cbc None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:24.809937 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291302e-8a20-4fc4-93c4-fb32d3bc877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:24.810592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291302e-8a20-4fc4-93c4-fb32d3bc877a None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:24.810875 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291302e-8a20-4fc4-93c4-fb32d3bc877a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:24.811152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291302e-8a20-4fc4-93c4-fb32d3bc877a None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:24.811847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291302e-8a20-4fc4-93c4-fb32d3bc877a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:24.813248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291302e-8a20-4fc4-93c4-fb32d3bc877a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:24.842238 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 415/839] 10.222.0.33 () {60 vars in 1144 bytes} [Tue Oct 4 12:36:24 2022] POST /identity/v3/endpoints => generated 337 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:36:25.020548 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fa797-6625-4a3d-ab2c-630134f60cbc None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:25.111326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180eb02-0478-4d1a-9d16-d3fa87a93331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:25.112421 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180eb02-0478-4d1a-9d16-d3fa87a93331 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:25.112896 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180eb02-0478-4d1a-9d16-d3fa87a93331 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:25.113385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180eb02-0478-4d1a-9d16-d3fa87a93331 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:25.115005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180eb02-0478-4d1a-9d16-d3fa87a93331 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:25.115581 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180eb02-0478-4d1a-9d16-d3fa87a93331 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:25.119235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180eb02-0478-4d1a-9d16-d3fa87a93331 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:25.157537 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:25.165958 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 425/840] 10.222.0.33 () {56 vars in 1149 bytes} [Tue Oct 4 12:36:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:25.175227 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291302e-8a20-4fc4-93c4-fb32d3bc877a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:25.259757 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af532e62-1fcd-4dac-a659-ca52a51688f3 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:25.260053 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af532e62-1fcd-4dac-a659-ca52a51688f3 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:25.261386 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af532e62-1fcd-4dac-a659-ca52a51688f3 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:25.263225 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 426/841] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:25.277596 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e166f-2e16-43d0-906b-185a70150a78 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:25.277958 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e166f-2e16-43d0-906b-185a70150a78 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:25.278258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e166f-2e16-43d0-906b-185a70150a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:25.303830 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-d59e166f-2e16-43d0-906b-185a70150a78 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:25.305231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-d59e166f-2e16-43d0-906b-185a70150a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:25.377438 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:25.424996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291302e-8a20-4fc4-93c4-fb32d3bc877a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:25.486314 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 427/842] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:25.489949 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61571a3-355f-4109-8fc2-56fc73a12f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:25.491006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61571a3-355f-4109-8fc2-56fc73a12f80 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:25.491493 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61571a3-355f-4109-8fc2-56fc73a12f80 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:25.491948 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61571a3-355f-4109-8fc2-56fc73a12f80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:25.625174 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61571a3-355f-4109-8fc2-56fc73a12f80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:25.627927 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61571a3-355f-4109-8fc2-56fc73a12f80 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:25.780532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:25.817470 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 416/843] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:36:25 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:36:25.825947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e166f-2e16-43d0-906b-185a70150a78 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:25.971074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61571a3-355f-4109-8fc2-56fc73a12f80 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:26.015906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e166f-2e16-43d0-906b-185a70150a78 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:26.063173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:26.076446 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5cdb0-7c57-40dc-bfc5-71cdb2f44c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:26.077624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5cdb0-7c57-40dc-bfc5-71cdb2f44c8d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:26.078136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5cdb0-7c57-40dc-bfc5-71cdb2f44c8d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:26.078612 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5cdb0-7c57-40dc-bfc5-71cdb2f44c8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:26.099189 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5cdb0-7c57-40dc-bfc5-71cdb2f44c8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:26.102144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5cdb0-7c57-40dc-bfc5-71cdb2f44c8d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:26.125648 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61571a3-355f-4109-8fc2-56fc73a12f80 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-61c5cdb0-7c57-40dc-bfc5-71cdb2f44c8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:26.132006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:26.134418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:26.136136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:26.136136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:26.136136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:26.136136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 428/844] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:26.137259 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c2bbe-938f-4d45-9a0c-625035431acd None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:26.145217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c2bbe-938f-4d45-9a0c-625035431acd None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:26.145217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c2bbe-938f-4d45-9a0c-625035431acd None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:26.145217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 429/845] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:26.145217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c2bbe-938f-4d45-9a0c-625035431acd None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:26.177520 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14356-cb3d-4b8c-b32b-c2fba98fdc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:26.178193 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14356-cb3d-4b8c-b32b-c2fba98fdc2b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:26.178473 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14356-cb3d-4b8c-b32b-c2fba98fdc2b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:26.178924 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14356-cb3d-4b8c-b32b-c2fba98fdc2b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:26.179517 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14356-cb3d-4b8c-b32b-c2fba98fdc2b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:26.181887 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14356-cb3d-4b8c-b32b-c2fba98fdc2b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-77c14356-cb3d-4b8c-b32b-c2fba98fdc2b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:26.191945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:26.193485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:26.195152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 417/846] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:36:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:26.197154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617aa2e9-0312-4ae2-b63b-4063566666c9 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:26.197154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617aa2e9-0312-4ae2-b63b-4063566666c9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:26.197154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617aa2e9-0312-4ae2-b63b-4063566666c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:26.218720 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-617aa2e9-0312-4ae2-b63b-4063566666c9 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:26.219942 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-617aa2e9-0312-4ae2-b63b-4063566666c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:26.281532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:26.329916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 418/847] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:26.331505 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617aa2e9-0312-4ae2-b63b-4063566666c9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:26.357451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c2bbe-938f-4d45-9a0c-625035431acd None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:26.426299 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be069b82-91b4-4371-a025-e1954ac952d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:26.427315 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be069b82-91b4-4371-a025-e1954ac952d9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:26.427803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be069b82-91b4-4371-a025-e1954ac952d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:26.428266 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be069b82-91b4-4371-a025-e1954ac952d9 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:26.429411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be069b82-91b4-4371-a025-e1954ac952d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:26.429947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be069b82-91b4-4371-a025-e1954ac952d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:26.433462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be069b82-91b4-4371-a025-e1954ac952d9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:26.456606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 430/848] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:26.465816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be069b82-91b4-4371-a025-e1954ac952d9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:26.516276 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617aa2e9-0312-4ae2-b63b-4063566666c9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:26.547021 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:26.580563 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc8b73-1c38-4810-9340-b5d4aef1c714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:26.582197 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc8b73-1c38-4810-9340-b5d4aef1c714 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:26.582907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc8b73-1c38-4810-9340-b5d4aef1c714 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:26.583641 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc8b73-1c38-4810-9340-b5d4aef1c714 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:26.585147 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc8b73-1c38-4810-9340-b5d4aef1c714 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:26.585929 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc8b73-1c38-4810-9340-b5d4aef1c714 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:26.588844 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc8b73-1c38-4810-9340-b5d4aef1c714 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:26.602928 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 419/849] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:36:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:26.606831 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc8b73-1c38-4810-9340-b5d4aef1c714 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:26.661783 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be069b82-91b4-4371-a025-e1954ac952d9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:26.685180 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:26.723450 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b29f39-78df-4c53-960c-2cf1c37aec5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:26.724454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b29f39-78df-4c53-960c-2cf1c37aec5e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:26.724916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b29f39-78df-4c53-960c-2cf1c37aec5e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:26.725365 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b29f39-78df-4c53-960c-2cf1c37aec5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:26.738667 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b29f39-78df-4c53-960c-2cf1c37aec5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:26.741928 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b29f39-78df-4c53-960c-2cf1c37aec5e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:26.757572 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc8b73-1c38-4810-9340-b5d4aef1c714 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-e5b29f39-78df-4c53-960c-2cf1c37aec5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:26.766584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:26.768488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:26.771982 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:26.771982 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:26.771982 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:26.771982 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:26.771982 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:26.771982 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:26.771982 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:26.771982 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:26.775018 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 431/850] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:26.775941 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b29f39-78df-4c53-960c-2cf1c37aec5e None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:26.809801 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839432fa-6764-4a86-99c1-d67628878241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:26.811037 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839432fa-6764-4a86-99c1-d67628878241 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:26.811321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839432fa-6764-4a86-99c1-d67628878241 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:26.811601 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839432fa-6764-4a86-99c1-d67628878241 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:26.812248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839432fa-6764-4a86-99c1-d67628878241 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:26.812550 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839432fa-6764-4a86-99c1-d67628878241 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:26.814274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839432fa-6764-4a86-99c1-d67628878241 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:26.822848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 420/851] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:36:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:26.824530 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839432fa-6764-4a86-99c1-d67628878241 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:26.987060 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839432fa-6764-4a86-99c1-d67628878241 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:26.989212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:27.010512 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b29f39-78df-4c53-960c-2cf1c37aec5e None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:27.012575 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:27.045872 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48bdf59-2deb-445c-9119-39eaf28b30ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:27.046414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bdf59-2deb-445c-9119-39eaf28b30ad None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:27.046696 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bdf59-2deb-445c-9119-39eaf28b30ad None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:27.046968 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bdf59-2deb-445c-9119-39eaf28b30ad None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:27.047687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48bdf59-2deb-445c-9119-39eaf28b30ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:27.047990 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48bdf59-2deb-445c-9119-39eaf28b30ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:27.049483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48bdf59-2deb-445c-9119-39eaf28b30ad None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:27.066736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 421/852] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:27.077975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48bdf59-2deb-445c-9119-39eaf28b30ad None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:27.086619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80205a-612f-4f4f-b856-feeee5e006f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:27.087392 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80205a-612f-4f4f-b856-feeee5e006f5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:27.087674 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80205a-612f-4f4f-b856-feeee5e006f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:27.087950 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80205a-612f-4f4f-b856-feeee5e006f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:27.100556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80205a-612f-4f4f-b856-feeee5e006f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-9b80205a-612f-4f4f-b856-feeee5e006f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:27.106803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:27.108844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:27.110601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:27.110601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:27.110601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:27.110601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:27.110601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:27.110601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:27.110601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:27.110601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:27.110601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:27.110601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:27.110601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:36:27.110601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:27.112963 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 432/853] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:36:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:27.114329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80205a-612f-4f4f-b856-feeee5e006f5 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:27.230132 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48bdf59-2deb-445c-9119-39eaf28b30ad None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:27.275819 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028f80b-7828-46f5-ab80-cd7de7c16191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:27.277188 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028f80b-7828-46f5-ab80-cd7de7c16191 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:27.277949 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028f80b-7828-46f5-ab80-cd7de7c16191 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:27.278416 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028f80b-7828-46f5-ab80-cd7de7c16191 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:27.279486 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028f80b-7828-46f5-ab80-cd7de7c16191 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=58d82737c2864faf8fef3b53167c4dfc) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:27.280056 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028f80b-7828-46f5-ab80-cd7de7c16191 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:27.281659 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028f80b-7828-46f5-ab80-cd7de7c16191 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:27.286486 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80205a-612f-4f4f-b856-feeee5e006f5 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:27.290936 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 422/854] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:36:27 2022] GET /identity/v3/role_assignments?user.id=58d82737c2864faf8fef3b53167c4dfc&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:27.335976 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759380e-710e-458a-9179-0466ae9b6945 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:27.336247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759380e-710e-458a-9179-0466ae9b6945 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:27.336416 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759380e-710e-458a-9179-0466ae9b6945 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:27.338332 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 423/855] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:36:27.344237 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c83a5e-766b-47d3-a3ed-1df110040c7b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:27.344543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c83a5e-766b-47d3-a3ed-1df110040c7b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:27.344773 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c83a5e-766b-47d3-a3ed-1df110040c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:27.368594 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-70c83a5e-766b-47d3-a3ed-1df110040c7b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:27.369924 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-70c83a5e-766b-47d3-a3ed-1df110040c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:27.374638 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e9a65-15c7-465b-a5dd-e50c7f476ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:27.375234 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e9a65-15c7-465b-a5dd-e50c7f476ee2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:27.375507 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e9a65-15c7-465b-a5dd-e50c7f476ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:27.375778 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e9a65-15c7-465b-a5dd-e50c7f476ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:27.376473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e9a65-15c7-465b-a5dd-e50c7f476ee2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:27.376774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e9a65-15c7-465b-a5dd-e50c7f476ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:27.378247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e9a65-15c7-465b-a5dd-e50c7f476ee2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:27.388750 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 433/856] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:36:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:27.398621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e9a65-15c7-465b-a5dd-e50c7f476ee2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:27.417126 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:27.465474 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:27.517293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 424/857] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:27.631217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e9a65-15c7-465b-a5dd-e50c7f476ee2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:27.704181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b38dbc-9647-4182-908a-ccb6c8e6a75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:27.705795 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b38dbc-9647-4182-908a-ccb6c8e6a75b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:27.706734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b38dbc-9647-4182-908a-ccb6c8e6a75b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:27.707523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b38dbc-9647-4182-908a-ccb6c8e6a75b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:27.729814 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b38dbc-9647-4182-908a-ccb6c8e6a75b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:27.732850 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b38dbc-9647-4182-908a-ccb6c8e6a75b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-b7b38dbc-9647-4182-908a-ccb6c8e6a75b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:27.762148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:27.764081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:27.765785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:27.765785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:27.765785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:27.769930 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 434/858] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:27.779779 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c83a5e-766b-47d3-a3ed-1df110040c7b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:27.831045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b38dbc-9647-4182-908a-ccb6c8e6a75b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:27.940395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c83a5e-766b-47d3-a3ed-1df110040c7b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:27.983023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:28.013755 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b38dbc-9647-4182-908a-ccb6c8e6a75b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:28.023281 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff53e05-0492-462b-b6a8-189359fc77e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:28.025665 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff53e05-0492-462b-b6a8-189359fc77e5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:28.026186 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff53e05-0492-462b-b6a8-189359fc77e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:28.026677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff53e05-0492-462b-b6a8-189359fc77e5 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:28.028435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff53e05-0492-462b-b6a8-189359fc77e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:28.029015 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff53e05-0492-462b-b6a8-189359fc77e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:28.032519 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff53e05-0492-462b-b6a8-189359fc77e5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:28.058344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37daf0-3d17-42a5-b75e-69b43bb64317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:28.059414 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37daf0-3d17-42a5-b75e-69b43bb64317 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:28.059920 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37daf0-3d17-42a5-b75e-69b43bb64317 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:28.060416 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37daf0-3d17-42a5-b75e-69b43bb64317 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:28.061222 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37daf0-3d17-42a5-b75e-69b43bb64317 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:28.062988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37daf0-3d17-42a5-b75e-69b43bb64317 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:28.065988 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 425/859] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:28.068513 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff53e05-0492-462b-b6a8-189359fc77e5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-6d37daf0-3d17-42a5-b75e-69b43bb64317 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:28.079708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:28.081535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 12:36:28.083136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:28.088470 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 435/860] 10.222.0.33 () {56 vars in 1132 bytes} [Tue Oct 4 12:36:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:28.098749 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37daf0-3d17-42a5-b75e-69b43bb64317 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:28.107662 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:28.231264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37daf0-3d17-42a5-b75e-69b43bb64317 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:28.272459 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff53e05-0492-462b-b6a8-189359fc77e5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:28.276944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e87e21-7b1d-4474-8202-f2df2195d814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:28.277863 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e87e21-7b1d-4474-8202-f2df2195d814 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:28.278385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e87e21-7b1d-4474-8202-f2df2195d814 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:28.278980 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e87e21-7b1d-4474-8202-f2df2195d814 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:28.279856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e87e21-7b1d-4474-8202-f2df2195d814 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:28.280375 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e87e21-7b1d-4474-8202-f2df2195d814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:28.281997 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e87e21-7b1d-4474-8202-f2df2195d814 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:28.288078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 436/861] 10.222.0.33 () {56 vars in 1126 bytes} [Tue Oct 4 12:36:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:28.297157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e87e21-7b1d-4474-8202-f2df2195d814 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:28.302689 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:28.345418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea50168-cf59-4236-80da-529386515032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:28.346592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea50168-cf59-4236-80da-529386515032 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:28.347140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea50168-cf59-4236-80da-529386515032 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:28.347675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea50168-cf59-4236-80da-529386515032 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:28.362704 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea50168-cf59-4236-80da-529386515032 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:28.365323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea50168-cf59-4236-80da-529386515032 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-5ea50168-cf59-4236-80da-529386515032 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:28.390103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:28.391971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:28.394850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:28.394850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:28.394850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:28.394850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:28.394850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:28.394850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:28.394850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:28.394850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:28.394850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 12:36:28.394850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:28.396739 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 426/862] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:36:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:28.397818 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea50168-cf59-4236-80da-529386515032 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:28.465243 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e87e21-7b1d-4474-8202-f2df2195d814 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:28.537731 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:28.547437 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60785df7-255f-485b-aef3-9dc2f89c0352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:28.549449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60785df7-255f-485b-aef3-9dc2f89c0352 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:28.550008 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60785df7-255f-485b-aef3-9dc2f89c0352 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:28.551117 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60785df7-255f-485b-aef3-9dc2f89c0352 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:28.553297 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60785df7-255f-485b-aef3-9dc2f89c0352 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:28.554126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60785df7-255f-485b-aef3-9dc2f89c0352 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:28.557978 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60785df7-255f-485b-aef3-9dc2f89c0352 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:28.570010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 437/863] 10.222.0.33 () {56 vars in 1126 bytes} [Tue Oct 4 12:36:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:28.576347 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea50168-cf59-4236-80da-529386515032 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:28.641267 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc1893-10df-483e-9d6d-a62e930f9d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:28.642349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc1893-10df-483e-9d6d-a62e930f9d97 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:28.642804 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc1893-10df-483e-9d6d-a62e930f9d97 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:28.643277 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc1893-10df-483e-9d6d-a62e930f9d97 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:28.644781 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc1893-10df-483e-9d6d-a62e930f9d97 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:28.645297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc1893-10df-483e-9d6d-a62e930f9d97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:28.647120 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc1893-10df-483e-9d6d-a62e930f9d97 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:28.670165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 427/864] 10.222.0.33 () {56 vars in 1135 bytes} [Tue Oct 4 12:36:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:28.679342 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60785df7-255f-485b-aef3-9dc2f89c0352 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:28.783594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:28.845219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60785df7-255f-485b-aef3-9dc2f89c0352 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:28.891412 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1876b-807f-47b1-9c7a-623e8d3349f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:28.892586 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1876b-807f-47b1-9c7a-623e8d3349f3 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:28.893310 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1876b-807f-47b1-9c7a-623e8d3349f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:28.894127 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1876b-807f-47b1-9c7a-623e8d3349f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:29.007109 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1876b-807f-47b1-9c7a-623e8d3349f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:29.010344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1876b-807f-47b1-9c7a-623e8d3349f3 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:29.114425 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:29.161484 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 438/865] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:36:28 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:36:29.167597 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc1893-10df-483e-9d6d-a62e930f9d97 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:29.282093 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc1893-10df-483e-9d6d-a62e930f9d97 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:29.319711 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be9e3c-a0e2-4176-b368-c9a189c9e2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:29.320356 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be9e3c-a0e2-4176-b368-c9a189c9e2ab None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:29.320633 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be9e3c-a0e2-4176-b368-c9a189c9e2ab None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:29.320905 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be9e3c-a0e2-4176-b368-c9a189c9e2ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:29.323269 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:29.332575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be9e3c-a0e2-4176-b368-c9a189c9e2ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:29.334488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be9e3c-a0e2-4176-b368-c9a189c9e2ab None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c3be9e3c-a0e2-4176-b368-c9a189c9e2ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:29.353351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:29.354448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:29.355330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:29.355330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:29.355330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:29.355330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 428/866] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:29.363225 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1876b-807f-47b1-9c7a-623e8d3349f3 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:29.480822 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6b976-489c-4f52-9962-c1782e7160de None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:29.481017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6b976-489c-4f52-9962-c1782e7160de None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:29.481280 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6b976-489c-4f52-9962-c1782e7160de None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:29.482925 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 429/867] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:29.488588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cff79-5a4d-4eeb-b120-a0a223385c3c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:29.488883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cff79-5a4d-4eeb-b120-a0a223385c3c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:29.489138 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cff79-5a4d-4eeb-b120-a0a223385c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:29.513284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-190cff79-5a4d-4eeb-b120-a0a223385c3c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:29.515416 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-190cff79-5a4d-4eeb-b120-a0a223385c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:29.563277 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1876b-807f-47b1-9c7a-623e8d3349f3 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:29.614483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 430/868] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:29.637707 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e43925-f401-441f-8bb8-127cc0151005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:29.638898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e43925-f401-441f-8bb8-127cc0151005 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:29.639421 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e43925-f401-441f-8bb8-127cc0151005 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:29.639929 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e43925-f401-441f-8bb8-127cc0151005 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:29.641357 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e43925-f401-441f-8bb8-127cc0151005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:29.641921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e43925-f401-441f-8bb8-127cc0151005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:29.645298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e43925-f401-441f-8bb8-127cc0151005 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:29.671115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 439/869] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:29.676655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cff79-5a4d-4eeb-b120-a0a223385c3c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:29.805157 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cff79-5a4d-4eeb-b120-a0a223385c3c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:29.810186 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02eae3-cf5c-4817-ab96-d2b1ddc58540 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:29.810499 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02eae3-cf5c-4817-ab96-d2b1ddc58540 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:29.810742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02eae3-cf5c-4817-ab96-d2b1ddc58540 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:29.812370 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 440/870] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:29.818244 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b6c05-c568-4ea6-a226-014ea72bc04b None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:29.818576 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b6c05-c568-4ea6-a226-014ea72bc04b None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:29.818794 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b6c05-c568-4ea6-a226-014ea72bc04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:29.825848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:29.852130 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-8d3b6c05-c568-4ea6-a226-014ea72bc04b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:29.853726 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-8d3b6c05-c568-4ea6-a226-014ea72bc04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:29.859356 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863390e-7feb-4047-a1ee-427822f20f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:29.860008 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863390e-7feb-4047-a1ee-427822f20f61 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:29.860285 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863390e-7feb-4047-a1ee-427822f20f61 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:29.860557 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863390e-7feb-4047-a1ee-427822f20f61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:29.869954 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863390e-7feb-4047-a1ee-427822f20f61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:29.871573 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863390e-7feb-4047-a1ee-427822f20f61 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:29.884114 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-e863390e-7feb-4047-a1ee-427822f20f61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:29.885955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:29.886864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:29.887750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:29.887750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:29.887750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:29.887750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:29.887750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:29.887750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:29.887750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:29.887750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:29.887750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 431/871] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:29.894280 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863390e-7feb-4047-a1ee-427822f20f61 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:29.954803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 441/872] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:30.020510 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863390e-7feb-4047-a1ee-427822f20f61 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:30.060806 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd59f0f-3a11-4bf0-931a-a0dc5c0a9db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:30.061349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd59f0f-3a11-4bf0-931a-a0dc5c0a9db2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:30.061625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd59f0f-3a11-4bf0-931a-a0dc5c0a9db2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:30.061897 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd59f0f-3a11-4bf0-931a-a0dc5c0a9db2 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:30.062640 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd59f0f-3a11-4bf0-931a-a0dc5c0a9db2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:30.062945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd59f0f-3a11-4bf0-931a-a0dc5c0a9db2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:30.064403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd59f0f-3a11-4bf0-931a-a0dc5c0a9db2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:30.079544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 432/873] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:30.085498 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd59f0f-3a11-4bf0-931a-a0dc5c0a9db2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:30.086010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b6c05-c568-4ea6-a226-014ea72bc04b None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:30.157711 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:30.254904 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b6c05-c568-4ea6-a226-014ea72bc04b None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:30.268248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd59f0f-3a11-4bf0-931a-a0dc5c0a9db2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:30.309798 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea9d29-bc7b-4d0f-8aa1-43534edc821e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:30.311414 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea9d29-bc7b-4d0f-8aa1-43534edc821e None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:30.312206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea9d29-bc7b-4d0f-8aa1-43534edc821e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:30.312961 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea9d29-bc7b-4d0f-8aa1-43534edc821e None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:30.314473 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea9d29-bc7b-4d0f-8aa1-43534edc821e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:30.315997 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53313054-ed5c-435a-bcd9-f8654bcc761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:30.316614 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53313054-ed5c-435a-bcd9-f8654bcc761b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:30.316891 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53313054-ed5c-435a-bcd9-f8654bcc761b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:30.317167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53313054-ed5c-435a-bcd9-f8654bcc761b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:30.317900 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53313054-ed5c-435a-bcd9-f8654bcc761b None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=332fdff1d02a4664923ddbe0478eaa7a) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:30.318315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53313054-ed5c-435a-bcd9-f8654bcc761b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:30.318924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea9d29-bc7b-4d0f-8aa1-43534edc821e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:30.319753 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53313054-ed5c-435a-bcd9-f8654bcc761b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:30.325964 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 433/874] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:36:30 2022] GET /identity/v3/role_assignments?user.id=332fdff1d02a4664923ddbe0478eaa7a&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=fc68d072459d4720b9666c93bd535d63 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:30.326810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53313054-ed5c-435a-bcd9-f8654bcc761b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:30.353236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 442/875] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:36:30 2022] POST /identity/v3/services => generated 251 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:36:30.558701 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:30.562756 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53313054-ed5c-435a-bcd9-f8654bcc761b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:30.641046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdb23a-7e35-4362-b79b-c43ee7fd414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:30.642125 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdb23a-7e35-4362-b79b-c43ee7fd414b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:30.642681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdb23a-7e35-4362-b79b-c43ee7fd414b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:30.643177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdb23a-7e35-4362-b79b-c43ee7fd414b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:30.663458 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdb23a-7e35-4362-b79b-c43ee7fd414b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-bfcdb23a-7e35-4362-b79b-c43ee7fd414b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:30.674832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:30.676027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:30.677255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 434/876] 10.222.0.33 () {56 vars in 1120 bytes} [Tue Oct 4 12:36:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:30.682771 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea9d29-bc7b-4d0f-8aa1-43534edc821e None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:30.704730 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:30.816523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea9d29-bc7b-4d0f-8aa1-43534edc821e None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:30.856212 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da72b3a-05ef-46fd-92be-60ea30b877fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:30.857198 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da72b3a-05ef-46fd-92be-60ea30b877fb None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:30.857676 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da72b3a-05ef-46fd-92be-60ea30b877fb None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:30.858221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da72b3a-05ef-46fd-92be-60ea30b877fb None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:30.859112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da72b3a-05ef-46fd-92be-60ea30b877fb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:30.859697 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da72b3a-05ef-46fd-92be-60ea30b877fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:30.861363 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da72b3a-05ef-46fd-92be-60ea30b877fb None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:30.871637 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 443/877] 10.222.0.33 () {56 vars in 1115 bytes} [Tue Oct 4 12:36:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:30.880461 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdb23a-7e35-4362-b79b-c43ee7fd414b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:31.026334 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdb23a-7e35-4362-b79b-c43ee7fd414b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:31.065019 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8804eb-e972-4eff-968a-654766f8f9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:31.066155 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8804eb-e972-4eff-968a-654766f8f9fc None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:31.066706 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8804eb-e972-4eff-968a-654766f8f9fc None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:31.067200 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8804eb-e972-4eff-968a-654766f8f9fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:31.082400 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8804eb-e972-4eff-968a-654766f8f9fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:31.084293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8804eb-e972-4eff-968a-654766f8f9fc None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-ad8804eb-e972-4eff-968a-654766f8f9fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:31.101600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:31.102721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:31.103618 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:31.103618 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:31.103618 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:31.105791 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 435/878] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:31.115126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da72b3a-05ef-46fd-92be-60ea30b877fb None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:31.186450 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf2a4e-2e18-44bd-b0e0-4b4be0de8d90 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:31.186802 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf2a4e-2e18-44bd-b0e0-4b4be0de8d90 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:31.187012 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf2a4e-2e18-44bd-b0e0-4b4be0de8d90 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:31.190152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 436/879] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:36:31.194984 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c064a-cc58-4daa-bef4-52a4e1384cb9 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:31.195297 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c064a-cc58-4daa-bef4-52a4e1384cb9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:31.195479 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c064a-cc58-4daa-bef4-52a4e1384cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:31.217635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:31.217871 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-410c064a-cc58-4daa-bef4-52a4e1384cb9 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:31.219247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-410c064a-cc58-4daa-bef4-52a4e1384cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:31.221969 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:31.256629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da72b3a-05ef-46fd-92be-60ea30b877fb None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:31.322402 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc9925-46c4-45d0-9cc6-2e346f02c880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:31.323680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc9925-46c4-45d0-9cc6-2e346f02c880 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:31.324159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc9925-46c4-45d0-9cc6-2e346f02c880 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:31.324502 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc9925-46c4-45d0-9cc6-2e346f02c880 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:31.325579 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc9925-46c4-45d0-9cc6-2e346f02c880 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:31.326144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc9925-46c4-45d0-9cc6-2e346f02c880 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:31.328882 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc9925-46c4-45d0-9cc6-2e346f02c880 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:31.329229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 437/880] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:31.365207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 444/881] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:31.374498 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c064a-cc58-4daa-bef4-52a4e1384cb9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:31.545557 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c064a-cc58-4daa-bef4-52a4e1384cb9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:31.612411 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca44e63-f510-4ada-83de-b41be45a62d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:31.613437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca44e63-f510-4ada-83de-b41be45a62d8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:31.613901 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca44e63-f510-4ada-83de-b41be45a62d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:31.614394 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca44e63-f510-4ada-83de-b41be45a62d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:31.629133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca44e63-f510-4ada-83de-b41be45a62d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:31.631895 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca44e63-f510-4ada-83de-b41be45a62d8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:31.637366 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-8ca44e63-f510-4ada-83de-b41be45a62d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:31.659452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:31.661826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:31.664466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:31.664466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:31.664466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:31.664466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:31.664466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:31.664466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:31.664466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:31.664466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:31.664466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 12:36:31.664466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:31.664466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 438/882] 10.222.0.33 () {56 vars in 1117 bytes} [Tue Oct 4 12:36:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:31.667477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc9925-46c4-45d0-9cc6-2e346f02c880 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:31.671607 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca44e63-f510-4ada-83de-b41be45a62d8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:31.796828 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc9925-46c4-45d0-9cc6-2e346f02c880 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:31.836689 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b9a38-6844-4f9e-8c2f-6d9857364b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:31.837305 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b9a38-6844-4f9e-8c2f-6d9857364b0c None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:31.837584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b9a38-6844-4f9e-8c2f-6d9857364b0c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:31.837857 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b9a38-6844-4f9e-8c2f-6d9857364b0c None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:31.838560 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b9a38-6844-4f9e-8c2f-6d9857364b0c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:31.839945 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b9a38-6844-4f9e-8c2f-6d9857364b0c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:31.851010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:31.862364 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 445/883] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:36:31 2022] POST /identity/v3/services => generated 269 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:36:31.883552 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca44e63-f510-4ada-83de-b41be45a62d8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:31.968735 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7805025-6d20-4076-a8b1-7af48932b3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:31.969837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7805025-6d20-4076-a8b1-7af48932b3ba None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:31.970362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7805025-6d20-4076-a8b1-7af48932b3ba None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:31.970856 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7805025-6d20-4076-a8b1-7af48932b3ba None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:31.972294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7805025-6d20-4076-a8b1-7af48932b3ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:31.972834 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7805025-6d20-4076-a8b1-7af48932b3ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:31.975629 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7805025-6d20-4076-a8b1-7af48932b3ba None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:32.013302 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 439/884] 10.222.0.33 () {56 vars in 1149 bytes} [Tue Oct 4 12:36:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:32.022531 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b9a38-6844-4f9e-8c2f-6d9857364b0c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:32.140913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b9a38-6844-4f9e-8c2f-6d9857364b0c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:32.179277 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014bfb9-3881-4a96-9cc6-b1d546ab6913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:32.179899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014bfb9-3881-4a96-9cc6-b1d546ab6913 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:32.180178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014bfb9-3881-4a96-9cc6-b1d546ab6913 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:32.180487 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014bfb9-3881-4a96-9cc6-b1d546ab6913 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:32.256024 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014bfb9-3881-4a96-9cc6-b1d546ab6913 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:32.257593 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014bfb9-3881-4a96-9cc6-b1d546ab6913 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:32.270612 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:32.368575 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 446/885] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:36:32 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:36:32.374856 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7805025-6d20-4076-a8b1-7af48932b3ba None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:32.465987 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:32.543486 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7805025-6d20-4076-a8b1-7af48932b3ba None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:32.608689 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92115ab5-4809-40cd-8241-0f43db632cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:32.609705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92115ab5-4809-40cd-8241-0f43db632cce None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:32.610191 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92115ab5-4809-40cd-8241-0f43db632cce None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:32.610695 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92115ab5-4809-40cd-8241-0f43db632cce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:32.631030 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92115ab5-4809-40cd-8241-0f43db632cce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:32.633583 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92115ab5-4809-40cd-8241-0f43db632cce None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-92115ab5-4809-40cd-8241-0f43db632cce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:32.662072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:32.663096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:32.663967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:32.663967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:32.663967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:32.665152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 440/886] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:32.671214 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014bfb9-3881-4a96-9cc6-b1d546ab6913 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:32.817939 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:32.823282 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014bfb9-3881-4a96-9cc6-b1d546ab6913 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:32.856777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6188b6-1e57-4b57-91c7-4a77215ba716 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:32.857056 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6188b6-1e57-4b57-91c7-4a77215ba716 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:32.857391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6188b6-1e57-4b57-91c7-4a77215ba716 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:32.859037 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 441/887] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:36:32.865149 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efac61d-bc81-4423-b276-dd62c2213781 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:32.865466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efac61d-bc81-4423-b276-dd62c2213781 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:32.865788 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efac61d-bc81-4423-b276-dd62c2213781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:32.868993 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543c2f8-59f0-4d2f-88b8-6cc7129f1a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:32.869755 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543c2f8-59f0-4d2f-88b8-6cc7129f1a53 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:32.870123 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543c2f8-59f0-4d2f-88b8-6cc7129f1a53 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:32.870405 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543c2f8-59f0-4d2f-88b8-6cc7129f1a53 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:32.871316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543c2f8-59f0-4d2f-88b8-6cc7129f1a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:32.871483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543c2f8-59f0-4d2f-88b8-6cc7129f1a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:32.873632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543c2f8-59f0-4d2f-88b8-6cc7129f1a53 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:32.887408 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-2efac61d-bc81-4423-b276-dd62c2213781 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:32.888612 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2efac61d-bc81-4423-b276-dd62c2213781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:32.894244 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 447/888] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:32.903739 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543c2f8-59f0-4d2f-88b8-6cc7129f1a53 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:32.980565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 442/889] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:33.059746 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543c2f8-59f0-4d2f-88b8-6cc7129f1a53 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:33.065768 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250e7fd-bbcb-4d5d-a3fd-e458988c1470 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:33.066118 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250e7fd-bbcb-4d5d-a3fd-e458988c1470 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:33.066345 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250e7fd-bbcb-4d5d-a3fd-e458988c1470 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:33.068066 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 443/890] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:33.073989 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdfe30-64db-4a0b-bd50-808e0bb86de8 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:33.074320 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdfe30-64db-4a0b-bd50-808e0bb86de8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:33.074550 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdfe30-64db-4a0b-bd50-808e0bb86de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:33.096172 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-11bdfe30-64db-4a0b-bd50-808e0bb86de8 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:33.097504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-11bdfe30-64db-4a0b-bd50-808e0bb86de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:33.107419 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaee5bf-28f9-42fd-a49b-07e8c6f02686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:33.108924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaee5bf-28f9-42fd-a49b-07e8c6f02686 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:33.109695 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaee5bf-28f9-42fd-a49b-07e8c6f02686 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:33.110482 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaee5bf-28f9-42fd-a49b-07e8c6f02686 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:33.119457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaee5bf-28f9-42fd-a49b-07e8c6f02686 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:33.121352 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaee5bf-28f9-42fd-a49b-07e8c6f02686 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-cdaee5bf-28f9-42fd-a49b-07e8c6f02686 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:33.136614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:33.137588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:33.138521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:33.138521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:33.138521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:33.138521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:33.138521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:33.138521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:33.138521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:33.138521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:33.141080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 448/891] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:33.151174 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaee5bf-28f9-42fd-a49b-07e8c6f02686 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:33.159977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:33.183661 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:33.250077 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 444/892] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:33.258120 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdfe30-64db-4a0b-bd50-808e0bb86de8 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:33.301162 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaee5bf-28f9-42fd-a49b-07e8c6f02686 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:33.346510 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311c307-b972-4ad7-9e31-bafcf4b0bcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:33.347469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311c307-b972-4ad7-9e31-bafcf4b0bcf0 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:33.347997 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311c307-b972-4ad7-9e31-bafcf4b0bcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:33.348484 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311c307-b972-4ad7-9e31-bafcf4b0bcf0 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:33.349509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311c307-b972-4ad7-9e31-bafcf4b0bcf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:33.349994 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311c307-b972-4ad7-9e31-bafcf4b0bcf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:33.351717 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311c307-b972-4ad7-9e31-bafcf4b0bcf0 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:33.366020 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 449/893] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:33.376272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311c307-b972-4ad7-9e31-bafcf4b0bcf0 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:33.460902 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdfe30-64db-4a0b-bd50-808e0bb86de8 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:33.485080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:33.507010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311c307-b972-4ad7-9e31-bafcf4b0bcf0 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:33.533743 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38895068-3177-47c4-923c-7280869f1e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:33.535586 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38895068-3177-47c4-923c-7280869f1e79 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:33.536401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38895068-3177-47c4-923c-7280869f1e79 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:33.537255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38895068-3177-47c4-923c-7280869f1e79 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:33.538351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38895068-3177-47c4-923c-7280869f1e79 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:33.540958 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38895068-3177-47c4-923c-7280869f1e79 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:33.546809 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6b8bc-fcb7-405f-a49b-bbcc8e802d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:33.547325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6b8bc-fcb7-405f-a49b-bbcc8e802d67 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:33.547609 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6b8bc-fcb7-405f-a49b-bbcc8e802d67 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:33.547884 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6b8bc-fcb7-405f-a49b-bbcc8e802d67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:33.548618 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6b8bc-fcb7-405f-a49b-bbcc8e802d67 None admin] RBAC: Adding query filter params (role.id=0945a2902fad45818ce54d62ab8ff3eb, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=58d82737c2864faf8fef3b53167c4dfc) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:33.548928 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6b8bc-fcb7-405f-a49b-bbcc8e802d67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:33.550421 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6b8bc-fcb7-405f-a49b-bbcc8e802d67 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:33.557813 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 450/894] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:36:33 2022] GET /identity/v3/role_assignments?user.id=58d82737c2864faf8fef3b53167c4dfc&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=0945a2902fad45818ce54d62ab8ff3eb => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-38895068-3177-47c4-923c-7280869f1e79 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:33.558007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:33.559096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:33.561291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:33.562156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6b8bc-fcb7-405f-a49b-bbcc8e802d67 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:33.562230 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 445/895] 10.222.0.33 () {56 vars in 1126 bytes} [Tue Oct 4 12:36:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:33.570706 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38895068-3177-47c4-923c-7280869f1e79 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:33.606680 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:33.694710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38895068-3177-47c4-923c-7280869f1e79 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:33.697773 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6b8bc-fcb7-405f-a49b-bbcc8e802d67 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:33.733158 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e343cc7-cc04-46e2-b87f-919011d26f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:33.734121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e343cc7-cc04-46e2-b87f-919011d26f2a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:33.734678 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e343cc7-cc04-46e2-b87f-919011d26f2a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:33.735231 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e343cc7-cc04-46e2-b87f-919011d26f2a None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:33.736243 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e343cc7-cc04-46e2-b87f-919011d26f2a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:33.736829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e343cc7-cc04-46e2-b87f-919011d26f2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:33.738360 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66c008-6963-4092-b0c7-1a0ff51caf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:33.738827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e343cc7-cc04-46e2-b87f-919011d26f2a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:33.739173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66c008-6963-4092-b0c7-1a0ff51caf5c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:33.739526 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66c008-6963-4092-b0c7-1a0ff51caf5c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:33.739875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66c008-6963-4092-b0c7-1a0ff51caf5c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:33.740543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66c008-6963-4092-b0c7-1a0ff51caf5c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:33.742355 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66c008-6963-4092-b0c7-1a0ff51caf5c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:33.746595 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 446/896] 10.222.0.33 () {56 vars in 1122 bytes} [Tue Oct 4 12:36:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-6c66c008-6963-4092-b0c7-1a0ff51caf5c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:33.754891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:33.755933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:33.756742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 451/897] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:36:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:33.757528 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e343cc7-cc04-46e2-b87f-919011d26f2a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:33.765800 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66c008-6963-4092-b0c7-1a0ff51caf5c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:33.782965 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:33.913434 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:33.925266 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e343cc7-cc04-46e2-b87f-919011d26f2a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:33.944693 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66c008-6963-4092-b0c7-1a0ff51caf5c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:33.972440 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a59db2-d758-470e-8611-4d68c5905df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:33.973891 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a59db2-d758-470e-8611-4d68c5905df2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:33.974375 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a59db2-d758-470e-8611-4d68c5905df2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:33.974827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a59db2-d758-470e-8611-4d68c5905df2 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:33.975639 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a59db2-d758-470e-8611-4d68c5905df2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:33.976194 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a59db2-d758-470e-8611-4d68c5905df2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:33.978496 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a59db2-d758-470e-8611-4d68c5905df2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:33.984737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 447/898] 10.222.0.33 () {56 vars in 1122 bytes} [Tue Oct 4 12:36:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:34.010184 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad241fb-14b5-4ce6-b592-b5eadaa822dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:34.011191 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad241fb-14b5-4ce6-b592-b5eadaa822dc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:34.011687 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad241fb-14b5-4ce6-b592-b5eadaa822dc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:34.012142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad241fb-14b5-4ce6-b592-b5eadaa822dc None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:34.013000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad241fb-14b5-4ce6-b592-b5eadaa822dc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:34.013539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad241fb-14b5-4ce6-b592-b5eadaa822dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:34.015254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad241fb-14b5-4ce6-b592-b5eadaa822dc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:34.022116 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 452/899] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:36:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:34.035993 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a59db2-d758-470e-8611-4d68c5905df2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:34.234791 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a59db2-d758-470e-8611-4d68c5905df2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:34.299928 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42a90a-bc9f-43f1-9510-9aed9becd7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:34.300989 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42a90a-bc9f-43f1-9510-9aed9becd7e2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:34.301464 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42a90a-bc9f-43f1-9510-9aed9becd7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:34.301934 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42a90a-bc9f-43f1-9510-9aed9becd7e2 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:34.303183 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42a90a-bc9f-43f1-9510-9aed9becd7e2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:34.303693 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42a90a-bc9f-43f1-9510-9aed9becd7e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:34.306023 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42a90a-bc9f-43f1-9510-9aed9becd7e2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:34.322240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 448/900] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:36:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:34.326325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad241fb-14b5-4ce6-b592-b5eadaa822dc None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:34.403468 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cde217-b098-4123-a257-e9ca0a531f9d None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:34.403974 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cde217-b098-4123-a257-e9ca0a531f9d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:34.404478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cde217-b098-4123-a257-e9ca0a531f9d None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:34.407570 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 449/901] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:34.416143 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5539c59-c236-4031-8085-efe37df96b9c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:34.416523 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5539c59-c236-4031-8085-efe37df96b9c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:34.416772 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5539c59-c236-4031-8085-efe37df96b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:34.440279 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e5539c59-c236-4031-8085-efe37df96b9c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:34.441936 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e5539c59-c236-4031-8085-efe37df96b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:34.448445 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:34.462105 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:34.491033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad241fb-14b5-4ce6-b592-b5eadaa822dc None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:34.538079 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 450/902] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:34.543755 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd452822-e445-4855-b1e4-d4cfaf5c4967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:34.544502 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd452822-e445-4855-b1e4-d4cfaf5c4967 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:34.544896 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd452822-e445-4855-b1e4-d4cfaf5c4967 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:34.545344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd452822-e445-4855-b1e4-d4cfaf5c4967 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:34.546157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd452822-e445-4855-b1e4-d4cfaf5c4967 None admin] RBAC: Adding query filter params (interface=public, service_id=417dfe53cd8b4bc2a7a60932f7c6bfca, region_id=RegionOne) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:34.546526 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd452822-e445-4855-b1e4-d4cfaf5c4967 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:34.548066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd452822-e445-4855-b1e4-d4cfaf5c4967 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:34.555758 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 453/903] 10.222.0.33 () {56 vars in 1251 bytes} [Tue Oct 4 12:36:34 2022] GET /identity/v3/endpoints?service_id=417dfe53cd8b4bc2a7a60932f7c6bfca&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:34.564695 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5539c59-c236-4031-8085-efe37df96b9c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:34.716335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5539c59-c236-4031-8085-efe37df96b9c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:34.768851 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02535c4b-fe18-4e84-9f53-f2c1c5d40d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:34.769521 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02535c4b-fe18-4e84-9f53-f2c1c5d40d51 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:34.769794 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02535c4b-fe18-4e84-9f53-f2c1c5d40d51 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:34.770079 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02535c4b-fe18-4e84-9f53-f2c1c5d40d51 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:34.770872 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02535c4b-fe18-4e84-9f53-f2c1c5d40d51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:34.772611 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02535c4b-fe18-4e84-9f53-f2c1c5d40d51 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:34.780979 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 451/904] 10.222.0.33 () {56 vars in 1087 bytes} [Tue Oct 4 12:36:34 2022] GET /identity/v3/services => generated 1268 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:36:34.862778 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd452822-e445-4855-b1e4-d4cfaf5c4967 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:35.046728 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd452822-e445-4855-b1e4-d4cfaf5c4967 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:35.086221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e2fec-1e4a-4783-8366-529581def5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:35.086935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e2fec-1e4a-4783-8366-529581def5b9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:35.087215 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e2fec-1e4a-4783-8366-529581def5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:35.087489 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e2fec-1e4a-4783-8366-529581def5b9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:35.088112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e2fec-1e4a-4783-8366-529581def5b9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:35.089585 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e2fec-1e4a-4783-8366-529581def5b9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:35.093043 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-076e2fec-1e4a-4783-8366-529581def5b9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:35.100462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:35.103345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:35.105554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 454/905] 10.222.0.33 () {56 vars in 1132 bytes} [Tue Oct 4 12:36:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:35.110116 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02535c4b-fe18-4e84-9f53-f2c1c5d40d51 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:35.128582 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:35.226899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02535c4b-fe18-4e84-9f53-f2c1c5d40d51 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:35.266853 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ef484-b448-482b-8225-40356044db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:35.267770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ef484-b448-482b-8225-40356044db47 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:35.268276 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ef484-b448-482b-8225-40356044db47 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:35.269030 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ef484-b448-482b-8225-40356044db47 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:35.269885 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ef484-b448-482b-8225-40356044db47 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:35.270422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ef484-b448-482b-8225-40356044db47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:35.272025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ef484-b448-482b-8225-40356044db47 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:35.278107 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 452/906] 10.222.0.33 () {56 vars in 1126 bytes} [Tue Oct 4 12:36:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:35.287229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e2fec-1e4a-4783-8366-529581def5b9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:35.428752 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e2fec-1e4a-4783-8366-529581def5b9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:35.478602 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab460f48-17ee-4a93-80f6-cfc96459bcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:35.479258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab460f48-17ee-4a93-80f6-cfc96459bcdc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:35.479545 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab460f48-17ee-4a93-80f6-cfc96459bcdc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:35.479821 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab460f48-17ee-4a93-80f6-cfc96459bcdc None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:35.480483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab460f48-17ee-4a93-80f6-cfc96459bcdc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:35.480960 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab460f48-17ee-4a93-80f6-cfc96459bcdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:35.482861 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab460f48-17ee-4a93-80f6-cfc96459bcdc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:35.490023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 455/907] 10.222.0.33 () {56 vars in 1126 bytes} [Tue Oct 4 12:36:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:35.496664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ef484-b448-482b-8225-40356044db47 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:35.589497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:35.613356 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ef484-b448-482b-8225-40356044db47 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:35.651405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574416d-6151-4829-828a-d87bbeb33f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:35.652020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574416d-6151-4829-828a-d87bbeb33f1c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:35.652301 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574416d-6151-4829-828a-d87bbeb33f1c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:35.652578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574416d-6151-4829-828a-d87bbeb33f1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:35.653292 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574416d-6151-4829-828a-d87bbeb33f1c None admin] RBAC: Adding query filter params (interface=public, service_id=4a241f631b9640cab3e012001e755548, region_id=RegionOne) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:35.653602 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574416d-6151-4829-828a-d87bbeb33f1c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:35.655014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574416d-6151-4829-828a-d87bbeb33f1c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:35.661017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 453/908] 10.222.0.33 () {56 vars in 1251 bytes} [Tue Oct 4 12:36:35 2022] GET /identity/v3/endpoints?service_id=4a241f631b9640cab3e012001e755548&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:35.667988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab460f48-17ee-4a93-80f6-cfc96459bcdc None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:35.761406 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:35.815101 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab460f48-17ee-4a93-80f6-cfc96459bcdc None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:35.852766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa578e-6619-4535-9fd8-921cf8db63a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:35.853634 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa578e-6619-4535-9fd8-921cf8db63a8 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:35.854103 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa578e-6619-4535-9fd8-921cf8db63a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:35.854513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa578e-6619-4535-9fd8-921cf8db63a8 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:35.855252 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa578e-6619-4535-9fd8-921cf8db63a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:35.856812 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa578e-6619-4535-9fd8-921cf8db63a8 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:35.864148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 456/909] 10.222.0.33 () {56 vars in 1087 bytes} [Tue Oct 4 12:36:35 2022] GET /identity/v3/services => generated 1268 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:36:36.259740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a0f0d-bfc9-4ceb-be51-2195bfb3419d None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:36.260023 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a0f0d-bfc9-4ceb-be51-2195bfb3419d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:36.260294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a0f0d-bfc9-4ceb-be51-2195bfb3419d None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:36.261934 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 454/910] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:36.267547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13936855-e9dc-4e64-b1d1-d6f0a49cdfdc None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:36.267823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13936855-e9dc-4e64-b1d1-d6f0a49cdfdc None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:36.268091 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13936855-e9dc-4e64-b1d1-d6f0a49cdfdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:36.288840 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-13936855-e9dc-4e64-b1d1-d6f0a49cdfdc None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:36.290634 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-13936855-e9dc-4e64-b1d1-d6f0a49cdfdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:36.374537 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 457/911] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:36.483654 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d47e3-3dbb-42cc-b730-5ff47b0aee24 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:36.483941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d47e3-3dbb-42cc-b730-5ff47b0aee24 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:36.484308 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d47e3-3dbb-42cc-b730-5ff47b0aee24 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:36.485867 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 455/912] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:36.491833 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdfdd8-da9b-4aae-b884-0264a5634178 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:36.492600 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdfdd8-da9b-4aae-b884-0264a5634178 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:36.492600 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdfdd8-da9b-4aae-b884-0264a5634178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:36.526525 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-61cdfdd8-da9b-4aae-b884-0264a5634178 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:36.528570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-61cdfdd8-da9b-4aae-b884-0264a5634178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:36.631177 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:36.657517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 458/913] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:36.750305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d47e3-3dbb-42cc-b730-5ff47b0aee24 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:36.884616 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d47e3-3dbb-42cc-b730-5ff47b0aee24 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:36.929984 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccbb94-6370-476b-8576-b5e3042e4b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:36.930789 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccbb94-6370-476b-8576-b5e3042e4b55 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:36.931209 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccbb94-6370-476b-8576-b5e3042e4b55 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:36.931480 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccbb94-6370-476b-8576-b5e3042e4b55 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:36.944372 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccbb94-6370-476b-8576-b5e3042e4b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-b1ccbb94-6370-476b-8576-b5e3042e4b55 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:36.949381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:36.950389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:36.951240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 456/914] 10.222.0.33 () {56 vars in 1120 bytes} [Tue Oct 4 12:36:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:36.958814 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdfdd8-da9b-4aae-b884-0264a5634178 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:37.057348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccbb94-6370-476b-8576-b5e3042e4b55 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:37.064420 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:37.084655 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdfdd8-da9b-4aae-b884-0264a5634178 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:37.131156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fe8d0-e076-4ff7-9aab-5203b7fdb277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:37.131859 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fe8d0-e076-4ff7-9aab-5203b7fdb277 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:37.132149 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fe8d0-e076-4ff7-9aab-5203b7fdb277 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:37.132417 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fe8d0-e076-4ff7-9aab-5203b7fdb277 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:37.133087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fe8d0-e076-4ff7-9aab-5203b7fdb277 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:37.133961 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fe8d0-e076-4ff7-9aab-5203b7fdb277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:37.136415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fe8d0-e076-4ff7-9aab-5203b7fdb277 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:37.147070 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 459/915] 10.222.0.33 () {56 vars in 1115 bytes} [Tue Oct 4 12:36:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:37.153457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd93d4-1b0d-4de4-a0ad-123cbd387d14 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:37.153732 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd93d4-1b0d-4de4-a0ad-123cbd387d14 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:37.154010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd93d4-1b0d-4de4-a0ad-123cbd387d14 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:37.155537 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 460/916] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:37.156136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd93d4-1b0d-4de4-a0ad-123cbd387d14 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:37.189428 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccbb94-6370-476b-8576-b5e3042e4b55 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:37.210005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:37.232521 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7dc42-d345-44c4-885d-e95ef4b21096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:37.233557 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7dc42-d345-44c4-885d-e95ef4b21096 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:37.233557 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7dc42-d345-44c4-885d-e95ef4b21096 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:37.233831 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7dc42-d345-44c4-885d-e95ef4b21096 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:37.234344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7dc42-d345-44c4-885d-e95ef4b21096 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:37.235735 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7dc42-d345-44c4-885d-e95ef4b21096 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:37.262248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 457/917] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:36:37 2022] POST /identity/v3/services => generated 241 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:36:37.264845 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863ba1f-ec56-40ed-bbe6-1dcc5bd5df37 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:37.265427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863ba1f-ec56-40ed-bbe6-1dcc5bd5df37 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:37.265958 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863ba1f-ec56-40ed-bbe6-1dcc5bd5df37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:37.291127 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd93d4-1b0d-4de4-a0ad-123cbd387d14 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:37.292666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5863ba1f-ec56-40ed-bbe6-1dcc5bd5df37 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:37.295491 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5863ba1f-ec56-40ed-bbe6-1dcc5bd5df37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:37.337905 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3193ec-88fa-45dc-907e-e25a909e73c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:37.338814 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3193ec-88fa-45dc-907e-e25a909e73c9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:37.338997 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3193ec-88fa-45dc-907e-e25a909e73c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:37.339317 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3193ec-88fa-45dc-907e-e25a909e73c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:37.352583 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3193ec-88fa-45dc-907e-e25a909e73c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:37.354329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3193ec-88fa-45dc-907e-e25a909e73c9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-3c3193ec-88fa-45dc-907e-e25a909e73c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:37.373000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:37.374007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:37.374934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:37.374934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:37.374934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:37.374934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 461/918] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:37.379977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3193ec-88fa-45dc-907e-e25a909e73c9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:37.381384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:37.401571 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 458/919] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:37.573312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:37.580893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3193ec-88fa-45dc-907e-e25a909e73c9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:37.646525 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a499b48-ba74-48e6-98e3-16509bbddf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:37.647899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a499b48-ba74-48e6-98e3-16509bbddf55 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:37.648652 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a499b48-ba74-48e6-98e3-16509bbddf55 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:37.650617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a499b48-ba74-48e6-98e3-16509bbddf55 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:37.652096 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a499b48-ba74-48e6-98e3-16509bbddf55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:37.652840 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a499b48-ba74-48e6-98e3-16509bbddf55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:37.655709 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a499b48-ba74-48e6-98e3-16509bbddf55 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:37.681951 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 462/920] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:37.690090 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863ba1f-ec56-40ed-bbe6-1dcc5bd5df37 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:37.768879 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a499b48-ba74-48e6-98e3-16509bbddf55 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:37.839575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863ba1f-ec56-40ed-bbe6-1dcc5bd5df37 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:37.887738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9cd72-0ca4-4cb7-ade7-5ac65fc7f5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:37.888670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9cd72-0ca4-4cb7-ade7-5ac65fc7f5df None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:37.889111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9cd72-0ca4-4cb7-ade7-5ac65fc7f5df None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:37.889497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9cd72-0ca4-4cb7-ade7-5ac65fc7f5df None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:37.904188 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9cd72-0ca4-4cb7-ade7-5ac65fc7f5df None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:37.905957 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9cd72-0ca4-4cb7-ade7-5ac65fc7f5df None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-9be9cd72-0ca4-4cb7-ade7-5ac65fc7f5df None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:37.921627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:37.923776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:37.925975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:37.925975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:37.925975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:37.925975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:37.925975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:37.925975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:37.925975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:37.925975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:37.925975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 12:36:37.925975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:37.925975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 459/921] 10.222.0.33 () {56 vars in 1117 bytes} [Tue Oct 4 12:36:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:37.933131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9cd72-0ca4-4cb7-ade7-5ac65fc7f5df None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:38.020983 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a499b48-ba74-48e6-98e3-16509bbddf55 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:38.036442 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:38.089894 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283a8c3-504e-4508-99c6-25e7ee55ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:38.091719 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283a8c3-504e-4508-99c6-25e7ee55ccb3 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:38.092407 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283a8c3-504e-4508-99c6-25e7ee55ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:38.093228 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283a8c3-504e-4508-99c6-25e7ee55ccb3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:38.094740 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283a8c3-504e-4508-99c6-25e7ee55ccb3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:38.097616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283a8c3-504e-4508-99c6-25e7ee55ccb3 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-a283a8c3-504e-4508-99c6-25e7ee55ccb3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:38.115793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:38.118532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 12:36:38.120067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:38.129556 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9cd72-0ca4-4cb7-ade7-5ac65fc7f5df None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:38.130521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 463/922] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:36:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:38.135028 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283a8c3-504e-4508-99c6-25e7ee55ccb3 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:38.185199 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:38.194061 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ebe98-f358-431c-922e-68f7e8286c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:38.195126 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ebe98-f358-431c-922e-68f7e8286c29 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:38.195636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ebe98-f358-431c-922e-68f7e8286c29 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:38.196120 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ebe98-f358-431c-922e-68f7e8286c29 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:38.197473 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ebe98-f358-431c-922e-68f7e8286c29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:38.198024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ebe98-f358-431c-922e-68f7e8286c29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:38.201182 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ebe98-f358-431c-922e-68f7e8286c29 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:38.237316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 460/923] 10.222.0.33 () {56 vars in 1149 bytes} [Tue Oct 4 12:36:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:38.245684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ebe98-f358-431c-922e-68f7e8286c29 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:38.356414 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283a8c3-504e-4508-99c6-25e7ee55ccb3 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:38.373413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ebe98-f358-431c-922e-68f7e8286c29 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:38.385060 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:38.415546 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325ce3f-68aa-4bff-ad2e-53579990e2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:38.416204 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325ce3f-68aa-4bff-ad2e-53579990e2af None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:38.416481 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325ce3f-68aa-4bff-ad2e-53579990e2af None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:38.416748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325ce3f-68aa-4bff-ad2e-53579990e2af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:38.422354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed57700-e9a5-4e6d-878a-227ffc6ba269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:38.423383 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed57700-e9a5-4e6d-878a-227ffc6ba269 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:38.423880 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed57700-e9a5-4e6d-878a-227ffc6ba269 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:38.424380 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed57700-e9a5-4e6d-878a-227ffc6ba269 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:38.425527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed57700-e9a5-4e6d-878a-227ffc6ba269 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:38.426102 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed57700-e9a5-4e6d-878a-227ffc6ba269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:38.428302 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325ce3f-68aa-4bff-ad2e-53579990e2af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:38.428733 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed57700-e9a5-4e6d-878a-227ffc6ba269 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:38.429743 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325ce3f-68aa-4bff-ad2e-53579990e2af None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:38.442299 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 464/924] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:36:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:38.442299 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c9594-93b4-4741-9f89-216c3f958847 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:38.442299 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c9594-93b4-4741-9f89-216c3f958847 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:38.442299 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c9594-93b4-4741-9f89-216c3f958847 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:38.443037 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 465/925] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:38.444605 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c9594-93b4-4741-9f89-216c3f958847 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-e325ce3f-68aa-4bff-ad2e-53579990e2af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:38.453598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:38.454766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:38.456147 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:38.456147 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:38.456147 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:38.456147 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 461/926] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:38.456367 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5554b-0183-4b89-b5b0-ba72d8b26559 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:38.456615 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5554b-0183-4b89-b5b0-ba72d8b26559 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:38.459697 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5554b-0183-4b89-b5b0-ba72d8b26559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:38.484007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-bbb5554b-0183-4b89-b5b0-ba72d8b26559 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:38.485382 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bbb5554b-0183-4b89-b5b0-ba72d8b26559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:38.573536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 462/927] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:38.574264 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5554b-0183-4b89-b5b0-ba72d8b26559 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:38.584337 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:38.620483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c9594-93b4-4741-9f89-216c3f958847 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:38.701612 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd869e-f2ca-4981-8c37-543c3d78e283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:38.702411 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:38.703118 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd869e-f2ca-4981-8c37-543c3d78e283 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:38.703551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd869e-f2ca-4981-8c37-543c3d78e283 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:38.703928 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd869e-f2ca-4981-8c37-543c3d78e283 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:38.705025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd869e-f2ca-4981-8c37-543c3d78e283 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:38.705625 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd869e-f2ca-4981-8c37-543c3d78e283 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:38.707901 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd869e-f2ca-4981-8c37-543c3d78e283 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:38.715069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5554b-0183-4b89-b5b0-ba72d8b26559 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:38.715992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 466/928] 10.222.0.33 () {56 vars in 1114 bytes} [Tue Oct 4 12:36:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:38.721620 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd869e-f2ca-4981-8c37-543c3d78e283 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:38.764240 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e931dd8-0c92-4d33-b770-7f011a181621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:38.765268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e931dd8-0c92-4d33-b770-7f011a181621 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:38.765638 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e931dd8-0c92-4d33-b770-7f011a181621 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:38.765967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e931dd8-0c92-4d33-b770-7f011a181621 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:38.766810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e931dd8-0c92-4d33-b770-7f011a181621 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:38.767187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e931dd8-0c92-4d33-b770-7f011a181621 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:38.769080 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e931dd8-0c92-4d33-b770-7f011a181621 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:38.790074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 463/929] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:38.793627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e931dd8-0c92-4d33-b770-7f011a181621 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:38.890946 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd869e-f2ca-4981-8c37-543c3d78e283 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:38.933590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fb18b-92c3-4d31-89c5-f21d15676c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:38.934323 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fb18b-92c3-4d31-89c5-f21d15676c50 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:38.934567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fb18b-92c3-4d31-89c5-f21d15676c50 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:38.934813 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fb18b-92c3-4d31-89c5-f21d15676c50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:38.935496 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fb18b-92c3-4d31-89c5-f21d15676c50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:38.936918 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4fb18b-92c3-4d31-89c5-f21d15676c50 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:38.940837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e931dd8-0c92-4d33-b770-7f011a181621 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:38.964254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 467/930] 10.222.0.33 () {60 vars in 1144 bytes} [Tue Oct 4 12:36:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:36:38.964753 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fb18b-92c3-4d31-89c5-f21d15676c50 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:38.982517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:39.002079 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d21e02-e356-4b71-8686-55dea98c524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:39.002736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d21e02-e356-4b71-8686-55dea98c524b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:39.003056 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d21e02-e356-4b71-8686-55dea98c524b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:39.003331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d21e02-e356-4b71-8686-55dea98c524b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:39.012312 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d21e02-e356-4b71-8686-55dea98c524b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:39.013797 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d21e02-e356-4b71-8686-55dea98c524b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-e4d21e02-e356-4b71-8686-55dea98c524b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:39.026858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:39.028150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:39.029033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:39.029033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:39.029033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:39.029033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:39.029033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:39.029033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:39.029033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:39.029033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:39.029033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 464/931] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:39.037871 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d21e02-e356-4b71-8686-55dea98c524b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:39.054789 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:39.168697 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d21e02-e356-4b71-8686-55dea98c524b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:39.193236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fb18b-92c3-4d31-89c5-f21d15676c50 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:39.206857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535ea1d-a993-4dad-9be4-2d371a868c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:39.207490 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535ea1d-a993-4dad-9be4-2d371a868c97 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:39.207798 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535ea1d-a993-4dad-9be4-2d371a868c97 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:39.208072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535ea1d-a993-4dad-9be4-2d371a868c97 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:39.208770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535ea1d-a993-4dad-9be4-2d371a868c97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:39.209074 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535ea1d-a993-4dad-9be4-2d371a868c97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:39.211546 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535ea1d-a993-4dad-9be4-2d371a868c97 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:39.225112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 465/932] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:39.232750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535ea1d-a993-4dad-9be4-2d371a868c97 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:39.272100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54174f35-fe64-4a0b-ad67-58fde96e3396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:39.272901 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54174f35-fe64-4a0b-ad67-58fde96e3396 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:39.273024 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54174f35-fe64-4a0b-ad67-58fde96e3396 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:39.273294 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54174f35-fe64-4a0b-ad67-58fde96e3396 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:39.273929 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54174f35-fe64-4a0b-ad67-58fde96e3396 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:39.275498 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54174f35-fe64-4a0b-ad67-58fde96e3396 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-54174f35-fe64-4a0b-ad67-58fde96e3396 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:39.286665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:39.287792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:39.290027 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 468/933] 10.222.0.33 () {56 vars in 1132 bytes} [Tue Oct 4 12:36:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:39.294516 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54174f35-fe64-4a0b-ad67-58fde96e3396 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:39.358467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:39.394766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:39.401385 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535ea1d-a993-4dad-9be4-2d371a868c97 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:39.439979 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bbd5a-75ee-4328-97e3-e80d4c0b2324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:39.440910 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bbd5a-75ee-4328-97e3-e80d4c0b2324 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:39.441336 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bbd5a-75ee-4328-97e3-e80d4c0b2324 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:39.441782 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937bbd5a-75ee-4328-97e3-e80d4c0b2324 None admin] PATH_INFO: `/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/58d82737c2864faf8fef3b53167c4dfc/roles/0945a2902fad45818ce54d62ab8ff3eb` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:39.444296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937bbd5a-75ee-4328-97e3-e80d4c0b2324 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fb003a695490faa0077bbbe5e7d77, user_id=58d82737c2864faf8fef3b53167c4dfc, role_id=0945a2902fad45818ce54d62ab8ff3eb)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:39.445940 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937bbd5a-75ee-4328-97e3-e80d4c0b2324 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:39.462316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54174f35-fe64-4a0b-ad67-58fde96e3396 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:39.464050 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 466/934] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:36:39 2022] PUT /identity/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/58d82737c2864faf8fef3b53167c4dfc/roles/0945a2902fad45818ce54d62ab8ff3eb => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 12:36:39.515129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39f597-0b2c-4e46-848b-6e422dc8078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:39.516207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39f597-0b2c-4e46-848b-6e422dc8078c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:39.516712 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39f597-0b2c-4e46-848b-6e422dc8078c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:39.517167 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39f597-0b2c-4e46-848b-6e422dc8078c None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:39.518024 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39f597-0b2c-4e46-848b-6e422dc8078c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:39.518881 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39f597-0b2c-4e46-848b-6e422dc8078c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:39.520668 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39f597-0b2c-4e46-848b-6e422dc8078c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:39.531641 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 469/935] 10.222.0.33 () {56 vars in 1126 bytes} [Tue Oct 4 12:36:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:39.537951 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bbd5a-75ee-4328-97e3-e80d4c0b2324 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:39.672898 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937bbd5a-75ee-4328-97e3-e80d4c0b2324 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:39.710698 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:39.716238 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b8a26-3538-4789-b00e-a00ba55806f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:39.717624 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b8a26-3538-4789-b00e-a00ba55806f0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:39.717891 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b8a26-3538-4789-b00e-a00ba55806f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:39.718218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b8a26-3538-4789-b00e-a00ba55806f0 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:39.718874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b8a26-3538-4789-b00e-a00ba55806f0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:39.719181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b8a26-3538-4789-b00e-a00ba55806f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:39.720684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b8a26-3538-4789-b00e-a00ba55806f0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:39.727608 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 467/936] 10.222.0.33 () {56 vars in 1126 bytes} [Tue Oct 4 12:36:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:39.734240 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39f597-0b2c-4e46-848b-6e422dc8078c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:39.900187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39f597-0b2c-4e46-848b-6e422dc8078c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:39.909861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac58043-1e10-4ff6-87ef-c097fb73c082 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:39.911227 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac58043-1e10-4ff6-87ef-c097fb73c082 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:39.912043 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac58043-1e10-4ff6-87ef-c097fb73c082 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:39.913947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 468/937] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:39.921069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d03fad-c73f-4424-8631-43486d448be6 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:39.921719 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d03fad-c73f-4424-8631-43486d448be6 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:39.922727 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d03fad-c73f-4424-8631-43486d448be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:39.947813 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a2d03fad-c73f-4424-8631-43486d448be6 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:39.949472 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a2d03fad-c73f-4424-8631-43486d448be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:39.957453 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d798d-49de-4b86-9727-fe9a708d2790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:39.958637 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d798d-49de-4b86-9727-fe9a708d2790 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:39.959034 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d798d-49de-4b86-9727-fe9a708d2790 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:39.959449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d798d-49de-4b86-9727-fe9a708d2790 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:39.960540 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d798d-49de-4b86-9727-fe9a708d2790 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:39.963139 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d798d-49de-4b86-9727-fe9a708d2790 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:39.970389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:40.000358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 470/938] 10.222.0.33 () {60 vars in 1144 bytes} [Tue Oct 4 12:36:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:36:40.100980 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 469/939] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:40.382082 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d798d-49de-4b86-9727-fe9a708d2790 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:40.602518 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d798d-49de-4b86-9727-fe9a708d2790 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:40.684325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c9f6d-a871-4a41-a564-c0bdff44d7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:40.685881 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c9f6d-a871-4a41-a564-c0bdff44d7c6 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:40.686688 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c9f6d-a871-4a41-a564-c0bdff44d7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:40.687408 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c9f6d-a871-4a41-a564-c0bdff44d7c6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:40.688777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c9f6d-a871-4a41-a564-c0bdff44d7c6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:40.691535 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c9f6d-a871-4a41-a564-c0bdff44d7c6 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-813c9f6d-a871-4a41-a564-c0bdff44d7c6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:40.710734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:40.712501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:40.714245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:40.720095 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 471/940] 10.222.0.33 () {56 vars in 1126 bytes} [Tue Oct 4 12:36:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:40.728052 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d03fad-c73f-4424-8631-43486d448be6 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:41.017650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d03fad-c73f-4424-8631-43486d448be6 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:41.044909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:41.087481 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512659a-724a-4aff-9541-c71aacee7517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:41.088206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512659a-724a-4aff-9541-c71aacee7517 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:41.094070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512659a-724a-4aff-9541-c71aacee7517 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:41.094070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512659a-724a-4aff-9541-c71aacee7517 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:41.094070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512659a-724a-4aff-9541-c71aacee7517 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:41.094070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512659a-724a-4aff-9541-c71aacee7517 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:41.094070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512659a-724a-4aff-9541-c71aacee7517 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:41.111030 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 470/941] 10.222.0.33 () {56 vars in 1122 bytes} [Tue Oct 4 12:36:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:41.119504 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c9f6d-a871-4a41-a564-c0bdff44d7c6 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:41.209621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:41.275459 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c9f6d-a871-4a41-a564-c0bdff44d7c6 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:41.313954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153698ca-d148-432f-b1bb-dd60e1ae11e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:41.314978 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153698ca-d148-432f-b1bb-dd60e1ae11e4 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:41.315466 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153698ca-d148-432f-b1bb-dd60e1ae11e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:41.316001 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153698ca-d148-432f-b1bb-dd60e1ae11e4 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:41.316842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153698ca-d148-432f-b1bb-dd60e1ae11e4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:41.317350 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153698ca-d148-432f-b1bb-dd60e1ae11e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:41.318971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153698ca-d148-432f-b1bb-dd60e1ae11e4 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:41.325564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 472/942] 10.222.0.33 () {56 vars in 1122 bytes} [Tue Oct 4 12:36:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:41.335300 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512659a-724a-4aff-9541-c71aacee7517 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:41.442121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512659a-724a-4aff-9541-c71aacee7517 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:41.479150 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5759b1-7c2b-4abe-97ab-5a9a616d4e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:41.479773 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5759b1-7c2b-4abe-97ab-5a9a616d4e37 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:41.480051 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5759b1-7c2b-4abe-97ab-5a9a616d4e37 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:41.480318 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5759b1-7c2b-4abe-97ab-5a9a616d4e37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:41.481026 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5759b1-7c2b-4abe-97ab-5a9a616d4e37 None admin] RBAC: Adding query filter params (interface=public, service_id=58130b126a8a429687f911a532cee176, region_id=RegionOne) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:41.481334 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5759b1-7c2b-4abe-97ab-5a9a616d4e37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:41.482748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5759b1-7c2b-4abe-97ab-5a9a616d4e37 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:41.488982 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 471/943] 10.222.0.33 () {56 vars in 1251 bytes} [Tue Oct 4 12:36:41 2022] GET /identity/v3/endpoints?service_id=58130b126a8a429687f911a532cee176&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:41.498442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153698ca-d148-432f-b1bb-dd60e1ae11e4 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:41.629300 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153698ca-d148-432f-b1bb-dd60e1ae11e4 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:41.662466 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:41.667833 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2238b-c3d6-445a-a2c2-cacf7ad7bf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:41.669081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2238b-c3d6-445a-a2c2-cacf7ad7bf07 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:41.669745 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2238b-c3d6-445a-a2c2-cacf7ad7bf07 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:41.670254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2238b-c3d6-445a-a2c2-cacf7ad7bf07 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:41.671090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2238b-c3d6-445a-a2c2-cacf7ad7bf07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:41.672737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2238b-c3d6-445a-a2c2-cacf7ad7bf07 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:41.679858 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 473/944] 10.222.0.33 () {56 vars in 1087 bytes} [Tue Oct 4 12:36:41 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:36:42.184380 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e63a66-1ce1-403b-8619-0dadc9365a64 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:42.184906 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e63a66-1ce1-403b-8619-0dadc9365a64 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:42.184906 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e63a66-1ce1-403b-8619-0dadc9365a64 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:42.186683 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 472/945] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:36:42.192466 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97e48f-3d99-44b2-ae80-0c09ccdb4046 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:42.192947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97e48f-3d99-44b2-ae80-0c09ccdb4046 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:42.193025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97e48f-3d99-44b2-ae80-0c09ccdb4046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:42.212950 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-fc97e48f-3d99-44b2-ae80-0c09ccdb4046 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:42.214208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-fc97e48f-3d99-44b2-ae80-0c09ccdb4046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:42.222019 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09a3a3-0199-42a4-ba77-1435707fdacd None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:42.222358 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09a3a3-0199-42a4-ba77-1435707fdacd None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:42.222616 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09a3a3-0199-42a4-ba77-1435707fdacd None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:42.224262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 473/946] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:36:42.232423 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22136531-e37f-4b71-bd87-02dbeac1b9c2 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:42.233001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22136531-e37f-4b71-bd87-02dbeac1b9c2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:42.233321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22136531-e37f-4b71-bd87-02dbeac1b9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:42.257698 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-22136531-e37f-4b71-bd87-02dbeac1b9c2 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:42.257698 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-22136531-e37f-4b71-bd87-02dbeac1b9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:42.305660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:42.332340 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 474/947] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:42.350381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 474/948] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:42.435023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899e61b-d223-4045-bddf-17459d77a7a8 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:42.435616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899e61b-d223-4045-bddf-17459d77a7a8 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:42.436075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899e61b-d223-4045-bddf-17459d77a7a8 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:42.438115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 475/949] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:42.443801 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6050a-115c-4314-a3fc-4f6d7e750147 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:42.444079 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6050a-115c-4314-a3fc-4f6d7e750147 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:42.444354 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6050a-115c-4314-a3fc-4f6d7e750147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:42.466274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-68b6050a-115c-4314-a3fc-4f6d7e750147 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:42.467513 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-68b6050a-115c-4314-a3fc-4f6d7e750147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:42.541984 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 475/950] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:42.634227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899e61b-d223-4045-bddf-17459d77a7a8 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:42.733422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6050a-115c-4314-a3fc-4f6d7e750147 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:42.748444 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899e61b-d223-4045-bddf-17459d77a7a8 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:42.763610 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:42.786748 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f979765-5b17-4cf5-aa81-a1a25664e7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:42.787364 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f979765-5b17-4cf5-aa81-a1a25664e7a3 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:42.787648 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f979765-5b17-4cf5-aa81-a1a25664e7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:42.787920 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f979765-5b17-4cf5-aa81-a1a25664e7a3 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:42.788572 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f979765-5b17-4cf5-aa81-a1a25664e7a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:42.790083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f979765-5b17-4cf5-aa81-a1a25664e7a3 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:42.792704 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: WARNING keystone.server.flask.common [None req-5f979765-5b17-4cf5-aa81-a1a25664e7a3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 12:36:42.811517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.sql.core [None req-5f979765-5b17-4cf5-aa81-a1a25664e7a3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 12:36:42.811517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 12:36:42.811517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [parameters: {'id': 'd1da8206512c4bbc9bbf54d634804dc3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 12:36:42.811517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93092) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-5f979765-5b17-4cf5-aa81-a1a25664e7a3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d1da8206512c4bbc9bbf54d634804dc3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d1da8206512c4bbc9bbf54d634804dc3. Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application conn.query(q) Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 12:36:42.820328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application result.read() Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 04 12:36:42.821796 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application next(self.gen) Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application yield resource Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application next(self.gen) Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise self.value Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application session.commit() Oct 04 12:36:42.823007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self.session.flush() Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application compat.raise_( Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 12:36:42.824254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 12:36:42.825346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application conn.query(q) Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application result.read() Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 12:36:42.826289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [parameters: {'id': 'd1da8206512c4bbc9bbf54d634804dc3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:42.827396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:42.828209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:42.829088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:42.829088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 04 12:36:42.829088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 04 12:36:42.829088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:42.829088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:42.829088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 04 12:36:42.829088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 04 12:36:42.829088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d1da8206512c4bbc9bbf54d634804dc3. Oct 04 12:36:42.829088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:42.829088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 476/951] 10.222.0.33 () {60 vars in 1140 bytes} [Tue Oct 4 12:36:42 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:42.829088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f979765-5b17-4cf5-aa81-a1a25664e7a3 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:42.865332 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6050a-115c-4314-a3fc-4f6d7e750147 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:42.875130 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:42.909521 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb77dfc-8a77-41b1-bab8-736b494376a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:42.910305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb77dfc-8a77-41b1-bab8-736b494376a5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:42.910580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb77dfc-8a77-41b1-bab8-736b494376a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:42.910850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb77dfc-8a77-41b1-bab8-736b494376a5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:42.922130 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb77dfc-8a77-41b1-bab8-736b494376a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-afb77dfc-8a77-41b1-bab8-736b494376a5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:42.926992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:42.928112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:42.929278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:42.929278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:42.929278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:42.929278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:42.929278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:42.929278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:42.929278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:42.929278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:42.929278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:42.929278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:42.929278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 12:36:42.929278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:42.933435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 476/952] 10.222.0.33 () {56 vars in 1120 bytes} [Tue Oct 4 12:36:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:42.934411 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb77dfc-8a77-41b1-bab8-736b494376a5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:42.963724 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f979765-5b17-4cf5-aa81-a1a25664e7a3 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:43.004280 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5a1f3-d46f-4425-92c2-e425f64261c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:43.005089 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5a1f3-d46f-4425-92c2-e425f64261c7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:43.005384 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5a1f3-d46f-4425-92c2-e425f64261c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:43.005738 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5a1f3-d46f-4425-92c2-e425f64261c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:43.013567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5a1f3-d46f-4425-92c2-e425f64261c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:43.015357 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5a1f3-d46f-4425-92c2-e425f64261c7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-fdd5a1f3-d46f-4425-92c2-e425f64261c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:43.028906 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:43.029916 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:43.032284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:43.032284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:43.032284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:43.032284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:43.032284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:43.032284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:43.032284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:43.032284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:43.032284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 477/953] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:43.032284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5a1f3-d46f-4425-92c2-e425f64261c7 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:43.066243 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:43.070333 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb77dfc-8a77-41b1-bab8-736b494376a5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:43.114738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822a16c-6819-4ab6-9151-0aeb2935c114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:43.115388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822a16c-6819-4ab6-9151-0aeb2935c114 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:43.115666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822a16c-6819-4ab6-9151-0aeb2935c114 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:43.115941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822a16c-6819-4ab6-9151-0aeb2935c114 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:43.116525 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822a16c-6819-4ab6-9151-0aeb2935c114 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:43.119630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822a16c-6819-4ab6-9151-0aeb2935c114 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-4822a16c-6819-4ab6-9151-0aeb2935c114 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:43.130907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:43.132162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:43.133748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 477/954] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:43.135185 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822a16c-6819-4ab6-9151-0aeb2935c114 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:43.156383 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5a1f3-d46f-4425-92c2-e425f64261c7 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:43.174880 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:43.203526 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73450941-d7e2-4da7-8681-4d03d2059e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:43.205051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73450941-d7e2-4da7-8681-4d03d2059e1a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:43.205051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73450941-d7e2-4da7-8681-4d03d2059e1a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:43.205522 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73450941-d7e2-4da7-8681-4d03d2059e1a None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:43.206802 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73450941-d7e2-4da7-8681-4d03d2059e1a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:43.207351 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73450941-d7e2-4da7-8681-4d03d2059e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:43.209656 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73450941-d7e2-4da7-8681-4d03d2059e1a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:43.223005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 478/955] 10.222.0.33 () {56 vars in 1115 bytes} [Tue Oct 4 12:36:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:43.223907 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73450941-d7e2-4da7-8681-4d03d2059e1a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:43.285446 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822a16c-6819-4ab6-9151-0aeb2935c114 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:43.333171 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbf651-a72b-4dc4-b816-71a600e32b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:43.333797 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbf651-a72b-4dc4-b816-71a600e32b14 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:43.334098 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbf651-a72b-4dc4-b816-71a600e32b14 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:43.334373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbf651-a72b-4dc4-b816-71a600e32b14 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:43.335061 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbf651-a72b-4dc4-b816-71a600e32b14 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:43.335349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbf651-a72b-4dc4-b816-71a600e32b14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:43.338480 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbf651-a72b-4dc4-b816-71a600e32b14 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:43.339534 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:43.355770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73450941-d7e2-4da7-8681-4d03d2059e1a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:43.371683 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 478/956] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:43.375971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbf651-a72b-4dc4-b816-71a600e32b14 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:43.414792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe80fa0-aafc-4899-b5e9-6333bfc55081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:43.417456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe80fa0-aafc-4899-b5e9-6333bfc55081 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:43.418646 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe80fa0-aafc-4899-b5e9-6333bfc55081 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:43.419343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe80fa0-aafc-4899-b5e9-6333bfc55081 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:43.422056 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe80fa0-aafc-4899-b5e9-6333bfc55081 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:43.423060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe80fa0-aafc-4899-b5e9-6333bfc55081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:43.426988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe80fa0-aafc-4899-b5e9-6333bfc55081 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:43.440609 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 479/957] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:43.442219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe80fa0-aafc-4899-b5e9-6333bfc55081 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:43.530578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:43.544686 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbf651-a72b-4dc4-b816-71a600e32b14 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:43.592776 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe80fa0-aafc-4899-b5e9-6333bfc55081 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:43.598250 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419607e-7596-4153-9378-3bc05c08406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:43.598918 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419607e-7596-4153-9378-3bc05c08406c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:43.599196 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419607e-7596-4153-9378-3bc05c08406c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:43.599465 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419607e-7596-4153-9378-3bc05c08406c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:43.614293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419607e-7596-4153-9378-3bc05c08406c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:43.615802 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419607e-7596-4153-9378-3bc05c08406c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a419607e-7596-4153-9378-3bc05c08406c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:43.635732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:43.636795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:43.637783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:43.637783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:43.637783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:43.637783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 479/958] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:43.645530 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419607e-7596-4153-9378-3bc05c08406c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:43.656390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35636bd-f2c6-47d3-8854-4575bbf0ef9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:43.657289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35636bd-f2c6-47d3-8854-4575bbf0ef9c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:43.657560 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35636bd-f2c6-47d3-8854-4575bbf0ef9c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:43.657824 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35636bd-f2c6-47d3-8854-4575bbf0ef9c None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:43.658640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35636bd-f2c6-47d3-8854-4575bbf0ef9c None admin] RBAC: Adding query filter params (type=compute) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:43.658850 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35636bd-f2c6-47d3-8854-4575bbf0ef9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:43.660250 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35636bd-f2c6-47d3-8854-4575bbf0ef9c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:43.666296 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 480/959] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:43.789802 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419607e-7596-4153-9378-3bc05c08406c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:43.831419 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4efc0-0f59-4ab8-9492-73a42063b044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:43.832505 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4efc0-0f59-4ab8-9492-73a42063b044 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:43.832990 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4efc0-0f59-4ab8-9492-73a42063b044 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:43.833511 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4efc0-0f59-4ab8-9492-73a42063b044 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:43.834544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4efc0-0f59-4ab8-9492-73a42063b044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:43.835073 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4efc0-0f59-4ab8-9492-73a42063b044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:43.836730 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4efc0-0f59-4ab8-9492-73a42063b044 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:43.856933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 480/960] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:43.858904 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35636bd-f2c6-47d3-8854-4575bbf0ef9c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:43.883163 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:43.980943 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35636bd-f2c6-47d3-8854-4575bbf0ef9c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:44.023734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244ef9d-9e2b-4d7d-9098-7f8c44733ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:44.024747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244ef9d-9e2b-4d7d-9098-7f8c44733ba0 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:44.025236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244ef9d-9e2b-4d7d-9098-7f8c44733ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:44.025744 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244ef9d-9e2b-4d7d-9098-7f8c44733ba0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:44.034888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244ef9d-9e2b-4d7d-9098-7f8c44733ba0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:44.036702 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244ef9d-9e2b-4d7d-9098-7f8c44733ba0 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-b244ef9d-9e2b-4d7d-9098-7f8c44733ba0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:44.049932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:44.050969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:44.051931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:44.051931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:44.051931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:44.051931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:44.051931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:44.051931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:44.051931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:44.051931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:44.051931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 12:36:44.051931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:44.054670 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 481/961] 10.222.0.33 () {56 vars in 1117 bytes} [Tue Oct 4 12:36:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:44.064482 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4efc0-0f59-4ab8-9492-73a42063b044 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:44.092009 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:44.257793 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4efc0-0f59-4ab8-9492-73a42063b044 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:44.327119 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9116a-f220-4642-9785-dfac114f9638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:44.328165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9116a-f220-4642-9785-dfac114f9638 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:44.328710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9116a-f220-4642-9785-dfac114f9638 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:44.329191 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9116a-f220-4642-9785-dfac114f9638 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:44.330652 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9116a-f220-4642-9785-dfac114f9638 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:44.331268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9116a-f220-4642-9785-dfac114f9638 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:44.334173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9116a-f220-4642-9785-dfac114f9638 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:44.377722 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 481/962] 10.222.0.33 () {56 vars in 1149 bytes} [Tue Oct 4 12:36:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:44.379204 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244ef9d-9e2b-4d7d-9098-7f8c44733ba0 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:44.481474 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:44.524965 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244ef9d-9e2b-4d7d-9098-7f8c44733ba0 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:44.584349 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fe8fa-f5b3-4da4-94e0-5a65bc19c7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:44.585223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fe8fa-f5b3-4da4-94e0-5a65bc19c7f5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:44.585692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fe8fa-f5b3-4da4-94e0-5a65bc19c7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:44.586815 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fe8fa-f5b3-4da4-94e0-5a65bc19c7f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:44.708263 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fe8fa-f5b3-4da4-94e0-5a65bc19c7f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:44.710910 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fe8fa-f5b3-4da4-94e0-5a65bc19c7f5 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:44.829382 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:44.860204 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 482/963] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:36:44 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:36:44.870916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9116a-f220-4642-9785-dfac114f9638 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:44.984962 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:45.065998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9116a-f220-4642-9785-dfac114f9638 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:45.129066 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba0a62-3612-46d2-9e80-40cb1ae43076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:45.130651 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba0a62-3612-46d2-9e80-40cb1ae43076 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:45.131192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba0a62-3612-46d2-9e80-40cb1ae43076 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:45.131672 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba0a62-3612-46d2-9e80-40cb1ae43076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:45.150357 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffabb7-e9cb-4f2b-84fd-bad5e2d5b5b1 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:45.151318 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffabb7-e9cb-4f2b-84fd-bad5e2d5b5b1 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:45.151482 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba0a62-3612-46d2-9e80-40cb1ae43076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:45.152265 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffabb7-e9cb-4f2b-84fd-bad5e2d5b5b1 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:45.154197 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba0a62-3612-46d2-9e80-40cb1ae43076 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:45.155706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 483/964] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:45.166074 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab2730-c6df-45b1-8ba5-a3231c5f2743 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:45.166074 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab2730-c6df-45b1-8ba5-a3231c5f2743 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:45.166074 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab2730-c6df-45b1-8ba5-a3231c5f2743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-70ba0a62-3612-46d2-9e80-40cb1ae43076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:45.178239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:45.180692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:45.182397 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:45.182397 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:45.182397 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:45.186089 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-24ab2730-c6df-45b1-8ba5-a3231c5f2743 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:45.186382 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 482/965] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:45.186649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-24ab2730-c6df-45b1-8ba5-a3231c5f2743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:45.186894 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba0a62-3612-46d2-9e80-40cb1ae43076 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:45.287464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 484/966] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:45.362343 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba0a62-3612-46d2-9e80-40cb1ae43076 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:45.378425 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:45.399838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795cce2-a915-48b2-b5ba-59a2ede525ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:45.400349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795cce2-a915-48b2-b5ba-59a2ede525ac None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:45.400622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795cce2-a915-48b2-b5ba-59a2ede525ac None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:45.400902 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795cce2-a915-48b2-b5ba-59a2ede525ac None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:45.401545 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795cce2-a915-48b2-b5ba-59a2ede525ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:45.401845 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795cce2-a915-48b2-b5ba-59a2ede525ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:45.403312 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795cce2-a915-48b2-b5ba-59a2ede525ac None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:45.417711 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 483/967] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:45.426267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab2730-c6df-45b1-8ba5-a3231c5f2743 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:45.583258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795cce2-a915-48b2-b5ba-59a2ede525ac None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:45.587822 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab2730-c6df-45b1-8ba5-a3231c5f2743 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:45.598741 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:45.629841 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8d0c5-ddfb-4687-96cb-0ed724a18d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:45.630519 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8d0c5-ddfb-4687-96cb-0ed724a18d44 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:45.630797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8d0c5-ddfb-4687-96cb-0ed724a18d44 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:45.631069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8d0c5-ddfb-4687-96cb-0ed724a18d44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:45.638975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8d0c5-ddfb-4687-96cb-0ed724a18d44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:45.640450 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8d0c5-ddfb-4687-96cb-0ed724a18d44 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-f8f8d0c5-ddfb-4687-96cb-0ed724a18d44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:45.654874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:45.655788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:45.656933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:45.656933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:45.656933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:45.656933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:45.656933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:45.656933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:45.656933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:45.656933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:45.656933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 485/968] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:45.667160 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8d0c5-ddfb-4687-96cb-0ed724a18d44 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:45.794591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795cce2-a915-48b2-b5ba-59a2ede525ac None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:45.800273 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8d0c5-ddfb-4687-96cb-0ed724a18d44 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:45.858627 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7591e-e54b-4bff-aed9-e5ff039c4b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:45.859298 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7591e-e54b-4bff-aed9-e5ff039c4b12 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:45.859577 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7591e-e54b-4bff-aed9-e5ff039c4b12 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:45.859851 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7591e-e54b-4bff-aed9-e5ff039c4b12 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:45.860458 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7591e-e54b-4bff-aed9-e5ff039c4b12 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:45.861929 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7591e-e54b-4bff-aed9-e5ff039c4b12 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:45.867886 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c61a4-3ecd-44fe-b93b-63af9446ca4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:45.869264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c61a4-3ecd-44fe-b93b-63af9446ca4f None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:45.869856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c61a4-3ecd-44fe-b93b-63af9446ca4f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:45.870491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c61a4-3ecd-44fe-b93b-63af9446ca4f None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:45.872335 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c61a4-3ecd-44fe-b93b-63af9446ca4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:45.872706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c61a4-3ecd-44fe-b93b-63af9446ca4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-5ae7591e-e54b-4bff-aed9-e5ff039c4b12 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:45.875362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:45.876395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:45.877422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:45.878813 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c61a4-3ecd-44fe-b93b-63af9446ca4f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:45.878913 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 484/969] 10.222.0.33 () {56 vars in 1126 bytes} [Tue Oct 4 12:36:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:45.883104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7591e-e54b-4bff-aed9-e5ff039c4b12 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:45.904191 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 486/970] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:45.911010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c61a4-3ecd-44fe-b93b-63af9446ca4f None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:45.938154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:45.953577 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:46.029189 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7591e-e54b-4bff-aed9-e5ff039c4b12 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:46.038330 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c61a4-3ecd-44fe-b93b-63af9446ca4f None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:46.079144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114e3f2-51b1-40f8-b3b6-1098dad54b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:46.079806 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e3f2-51b1-40f8-b3b6-1098dad54b29 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:46.080089 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e3f2-51b1-40f8-b3b6-1098dad54b29 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:46.080361 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e3f2-51b1-40f8-b3b6-1098dad54b29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:46.081156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114e3f2-51b1-40f8-b3b6-1098dad54b29 None admin] RBAC: Adding query filter params (role.id=0945a2902fad45818ce54d62ab8ff3eb, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=58d82737c2864faf8fef3b53167c4dfc) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:46.081472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114e3f2-51b1-40f8-b3b6-1098dad54b29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:46.082094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2b1dc-44f6-4617-a3c8-693a6f1e4f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:46.082976 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114e3f2-51b1-40f8-b3b6-1098dad54b29 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:46.083078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2b1dc-44f6-4617-a3c8-693a6f1e4f22 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:46.083496 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2b1dc-44f6-4617-a3c8-693a6f1e4f22 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:46.084209 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2b1dc-44f6-4617-a3c8-693a6f1e4f22 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:46.085398 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2b1dc-44f6-4617-a3c8-693a6f1e4f22 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:46.085955 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2b1dc-44f6-4617-a3c8-693a6f1e4f22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:46.089168 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2b1dc-44f6-4617-a3c8-693a6f1e4f22 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:46.094304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 487/971] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:36:45 2022] GET /identity/v3/role_assignments?user.id=58d82737c2864faf8fef3b53167c4dfc&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=0945a2902fad45818ce54d62ab8ff3eb => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:46.096007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9e761-7a8e-4994-9dbd-81556b6708a4 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:46.096484 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9e761-7a8e-4994-9dbd-81556b6708a4 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:46.096923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9e761-7a8e-4994-9dbd-81556b6708a4 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:46.099149 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 488/972] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:46.107134 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278df3b0-2d78-4835-a738-625e84a71f6b None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:46.107404 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278df3b0-2d78-4835-a738-625e84a71f6b None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:46.108075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278df3b0-2d78-4835-a738-625e84a71f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:46.113522 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 485/973] 10.222.0.33 () {56 vars in 1122 bytes} [Tue Oct 4 12:36:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:46.120339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2b1dc-44f6-4617-a3c8-693a6f1e4f22 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:46.133809 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-278df3b0-2d78-4835-a738-625e84a71f6b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:46.135115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-278df3b0-2d78-4835-a738-625e84a71f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:46.238357 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 489/974] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:46.294137 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:46.298027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2b1dc-44f6-4617-a3c8-693a6f1e4f22 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:46.335538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70435c-5641-4f19-a0d2-3ced04e9a3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:46.336715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70435c-5641-4f19-a0d2-3ced04e9a3b3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:46.336993 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70435c-5641-4f19-a0d2-3ced04e9a3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:46.337264 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70435c-5641-4f19-a0d2-3ced04e9a3b3 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:46.337922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70435c-5641-4f19-a0d2-3ced04e9a3b3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:46.338245 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70435c-5641-4f19-a0d2-3ced04e9a3b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:46.339623 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70435c-5641-4f19-a0d2-3ced04e9a3b3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:46.346242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 486/975] 10.222.0.33 () {56 vars in 1122 bytes} [Tue Oct 4 12:36:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:46.352113 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278df3b0-2d78-4835-a738-625e84a71f6b None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:46.382394 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:46.472020 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278df3b0-2d78-4835-a738-625e84a71f6b None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:46.511017 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4db17-e5e3-4e3e-bed7-8d1e423e96dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:46.512041 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4db17-e5e3-4e3e-bed7-8d1e423e96dc None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:46.512525 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4db17-e5e3-4e3e-bed7-8d1e423e96dc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:46.512996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4db17-e5e3-4e3e-bed7-8d1e423e96dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:46.513881 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4db17-e5e3-4e3e-bed7-8d1e423e96dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:46.515509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4db17-e5e3-4e3e-bed7-8d1e423e96dc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:46.544785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 490/976] 10.222.0.33 () {60 vars in 1144 bytes} [Tue Oct 4 12:36:46 2022] POST /identity/v3/endpoints => generated 349 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:36:46.569034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70435c-5641-4f19-a0d2-3ced04e9a3b3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:46.684349 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4ca1a-764c-4bd5-8bbc-1e9ceb98f285 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:46.684696 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4ca1a-764c-4bd5-8bbc-1e9ceb98f285 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:46.684903 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4ca1a-764c-4bd5-8bbc-1e9ceb98f285 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:46.686653 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 491/977] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:46.692571 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a92c21-28a0-4f43-985f-a9584b93e093 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:46.692840 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a92c21-28a0-4f43-985f-a9584b93e093 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:46.693118 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a92c21-28a0-4f43-985f-a9584b93e093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:46.714186 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-61a92c21-28a0-4f43-985f-a9584b93e093 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:46.715516 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-61a92c21-28a0-4f43-985f-a9584b93e093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:46.754965 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70435c-5641-4f19-a0d2-3ced04e9a3b3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:46.814244 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3401d2-d2b4-4cd7-9629-60b5636876d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:46.814693 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3401d2-d2b4-4cd7-9629-60b5636876d6 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:46.818081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3401d2-d2b4-4cd7-9629-60b5636876d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:46.818081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3401d2-d2b4-4cd7-9629-60b5636876d6 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:46.818081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3401d2-d2b4-4cd7-9629-60b5636876d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:46.818903 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3401d2-d2b4-4cd7-9629-60b5636876d6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:46.819062 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 492/978] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:46.822273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: WARNING keystone.server.flask.common [None req-df3401d2-d2b4-4cd7-9629-60b5636876d6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 12:36:46.837349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.sql.core [None req-df3401d2-d2b4-4cd7-9629-60b5636876d6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 12:36:46.837349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 12:36:46.837349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [parameters: {'user_id': '7fafa7aca812444d99b94f961d7acc60', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 12:36:46.837349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93091) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-df3401d2-d2b4-4cd7-9629-60b5636876d6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application conn.query(q) Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application result.read() Oct 04 12:36:46.854996 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 12:36:46.856632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application next(self.gen) Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application yield resource Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application next(self.gen) Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise self.value Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application session.commit() Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 12:36:46.858666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self.session.flush() Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application compat.raise_( Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 12:36:46.860307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 12:36:46.863820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application conn.query(q) Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application result.read() Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 12:36:46.866058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [parameters: {'user_id': '7fafa7aca812444d99b94f961d7acc60', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:46.868741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 04 12:36:46.874478 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 04 12:36:46.875588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 04 12:36:46.875588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 04 12:36:46.875588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 12:36:46.875588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 12:36:46.875588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 12:36:46.875588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:46.875588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 487/979] 10.222.0.33 () {60 vars in 1131 bytes} [Tue Oct 4 12:36:46 2022] POST /identity/v3/users => generated 160 bytes in 289 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:46.875992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a92c21-28a0-4f43-985f-a9584b93e093 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:46.878020 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:46.989073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a92c21-28a0-4f43-985f-a9584b93e093 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:47.031974 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37debc9-81ff-4851-b5ec-6218adfb9da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:47.033036 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37debc9-81ff-4851-b5ec-6218adfb9da0 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:47.033515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37debc9-81ff-4851-b5ec-6218adfb9da0 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:47.034010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37debc9-81ff-4851-b5ec-6218adfb9da0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:47.043489 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37debc9-81ff-4851-b5ec-6218adfb9da0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:47.045397 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37debc9-81ff-4851-b5ec-6218adfb9da0 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-f37debc9-81ff-4851-b5ec-6218adfb9da0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:47.059934 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:47.060940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:47.061890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:47.061890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:47.061890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:47.061890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:47.061890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:47.061890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:47.061890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:47.061890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:47.061890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 12:36:47.061890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:47.069317 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 493/980] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:36:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:47.071484 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3401d2-d2b4-4cd7-9629-60b5636876d6 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:47.103855 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37debc9-81ff-4851-b5ec-6218adfb9da0 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:47.154660 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:47.211845 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3401d2-d2b4-4cd7-9629-60b5636876d6 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:47.221115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:47.259372 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bbfa3-d2e3-40d0-9e72-f60006ef5d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:47.260474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bbfa3-d2e3-40d0-9e72-f60006ef5d78 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:47.261197 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bbfa3-d2e3-40d0-9e72-f60006ef5d78 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:47.261721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bbfa3-d2e3-40d0-9e72-f60006ef5d78 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:47.263192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bbfa3-d2e3-40d0-9e72-f60006ef5d78 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:47.263724 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bbfa3-d2e3-40d0-9e72-f60006ef5d78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:47.265914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bbfa3-d2e3-40d0-9e72-f60006ef5d78 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:47.297196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37debc9-81ff-4851-b5ec-6218adfb9da0 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:47.330559 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 488/981] 10.222.0.33 () {56 vars in 1101 bytes} [Tue Oct 4 12:36:47 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:47.355529 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1fb282-87da-46e6-9647-d9c237753b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:47.356277 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fb282-87da-46e6-9647-d9c237753b50 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:47.356728 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fb282-87da-46e6-9647-d9c237753b50 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:47.357045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fb282-87da-46e6-9647-d9c237753b50 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:47.357941 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1fb282-87da-46e6-9647-d9c237753b50 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:47.360047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1fb282-87da-46e6-9647-d9c237753b50 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:47.384264 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 494/982] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:36:47 2022] POST /identity/v3/services => generated 242 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:36:48.675342 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8f68b-e9b7-4d8f-850d-e5c3ffa97bcd None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:48.675883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8f68b-e9b7-4d8f-850d-e5c3ffa97bcd None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:48.675883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8f68b-e9b7-4d8f-850d-e5c3ffa97bcd None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:48.677302 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 489/983] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:48.683233 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d72ae-6afe-4062-bdcd-e1c2bb839746 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:48.683689 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d72ae-6afe-4062-bdcd-e1c2bb839746 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:48.683763 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d72ae-6afe-4062-bdcd-e1c2bb839746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:48.703177 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-165d72ae-6afe-4062-bdcd-e1c2bb839746 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:48.704373 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-165d72ae-6afe-4062-bdcd-e1c2bb839746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:48.789913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 495/984] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:48.970536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c55495-8e15-4850-96a8-b2a261440209 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:48.970827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c55495-8e15-4850-96a8-b2a261440209 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:48.971014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c55495-8e15-4850-96a8-b2a261440209 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:48.972485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 490/985] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:48.978344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f2c5e-95e2-4bc6-a74c-b68728e23d5a None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:48.978631 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f2c5e-95e2-4bc6-a74c-b68728e23d5a None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:48.978905 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f2c5e-95e2-4bc6-a74c-b68728e23d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:48.998814 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-634f2c5e-95e2-4bc6-a74c-b68728e23d5a None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:49.000031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-634f2c5e-95e2-4bc6-a74c-b68728e23d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:49.049695 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:49.078081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 496/986] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:49.302752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c55495-8e15-4850-96a8-b2a261440209 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:49.344958 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f2c5e-95e2-4bc6-a74c-b68728e23d5a None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:49.435894 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:49.458193 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c55495-8e15-4850-96a8-b2a261440209 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:49.495805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7a206-b21a-42d3-987f-6b548878a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:49.496440 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7a206-b21a-42d3-987f-6b548878a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:49.496709 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7a206-b21a-42d3-987f-6b548878a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:49.496972 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7a206-b21a-42d3-987f-6b548878a3b8 None admin] PATH_INFO: `/v3/services/image` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:49.497554 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7a206-b21a-42d3-987f-6b548878a3b8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:49.499045 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7a206-b21a-42d3-987f-6b548878a3b8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:49.499469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f2c5e-95e2-4bc6-a74c-b68728e23d5a None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-52c7a206-b21a-42d3-987f-6b548878a3b8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:49.509339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:49.510384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:49.511219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 491/987] 10.222.0.33 () {56 vars in 1105 bytes} [Tue Oct 4 12:36:49 2022] GET /identity/v3/services/image => generated 86 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:49.520300 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7a206-b21a-42d3-987f-6b548878a3b8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:49.549337 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7aef0-9249-4072-bb6e-3c23b0ba57ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:49.550011 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7aef0-9249-4072-bb6e-3c23b0ba57ac None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:49.550325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7aef0-9249-4072-bb6e-3c23b0ba57ac None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:49.550602 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7aef0-9249-4072-bb6e-3c23b0ba57ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:49.551224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7aef0-9249-4072-bb6e-3c23b0ba57ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:49.552743 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7aef0-9249-4072-bb6e-3c23b0ba57ac None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-63e7aef0-9249-4072-bb6e-3c23b0ba57ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:49.563076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:49.565045 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:49.566614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 497/988] 10.222.0.33 () {56 vars in 1126 bytes} [Tue Oct 4 12:36:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:49.572326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7aef0-9249-4072-bb6e-3c23b0ba57ac None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:49.647828 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:49.706284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7a206-b21a-42d3-987f-6b548878a3b8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:49.714005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7aef0-9249-4072-bb6e-3c23b0ba57ac None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:49.748146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cc789-e03e-4285-91bd-25a3f0e7b6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:49.749177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cc789-e03e-4285-91bd-25a3f0e7b6db None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:49.749659 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cc789-e03e-4285-91bd-25a3f0e7b6db None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:49.750178 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cc789-e03e-4285-91bd-25a3f0e7b6db None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:49.750544 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876d5bf-56db-45c4-b175-19f308dcf863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:49.751140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876d5bf-56db-45c4-b175-19f308dcf863 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:49.751334 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670cc789-e03e-4285-91bd-25a3f0e7b6db None admin] RBAC: Adding query filter params (name=image) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:49.751432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876d5bf-56db-45c4-b175-19f308dcf863 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:49.751664 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876d5bf-56db-45c4-b175-19f308dcf863 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:49.751875 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670cc789-e03e-4285-91bd-25a3f0e7b6db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:49.752365 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876d5bf-56db-45c4-b175-19f308dcf863 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:49.752662 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876d5bf-56db-45c4-b175-19f308dcf863 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:49.754018 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876d5bf-56db-45c4-b175-19f308dcf863 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:49.754481 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670cc789-e03e-4285-91bd-25a3f0e7b6db None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:49.759472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 498/989] 10.222.0.33 () {56 vars in 1122 bytes} [Tue Oct 4 12:36:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:49.760656 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df314cd-ffc7-4d30-9680-a0b72e244609 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:49.760921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df314cd-ffc7-4d30-9680-a0b72e244609 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:49.761191 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df314cd-ffc7-4d30-9680-a0b72e244609 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:49.766074 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 499/990] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:49.766229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 492/991] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:49.766335 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df314cd-ffc7-4d30-9680-a0b72e244609 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:49.771215 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ed2e1-70b2-400c-a629-045c2d147bb7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:49.771740 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ed2e1-70b2-400c-a629-045c2d147bb7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:49.772011 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ed2e1-70b2-400c-a629-045c2d147bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:49.795447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-be6ed2e1-70b2-400c-a629-045c2d147bb7 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:49.796995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-be6ed2e1-70b2-400c-a629-045c2d147bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:49.799203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:49.881565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 493/992] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:49.882436 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ed2e1-70b2-400c-a629-045c2d147bb7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:49.903864 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df314cd-ffc7-4d30-9680-a0b72e244609 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:49.915428 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:49.955434 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd73bd9-a47a-413e-bd18-06ca0245481c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:49.956374 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd73bd9-a47a-413e-bd18-06ca0245481c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:49.956864 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd73bd9-a47a-413e-bd18-06ca0245481c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:49.957145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd73bd9-a47a-413e-bd18-06ca0245481c None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:49.958225 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd73bd9-a47a-413e-bd18-06ca0245481c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:49.958742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd73bd9-a47a-413e-bd18-06ca0245481c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:49.961477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd73bd9-a47a-413e-bd18-06ca0245481c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:49.969319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 500/993] 10.222.0.33 () {56 vars in 1122 bytes} [Tue Oct 4 12:36:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:49.977928 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd73bd9-a47a-413e-bd18-06ca0245481c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:50.018058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ed2e1-70b2-400c-a629-045c2d147bb7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:50.060209 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5a27a-7e14-47a4-88c5-ca621e9e43b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:50.060827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5a27a-7e14-47a4-88c5-ca621e9e43b1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:50.061101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5a27a-7e14-47a4-88c5-ca621e9e43b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:50.061365 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5a27a-7e14-47a4-88c5-ca621e9e43b1 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:50.062307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5a27a-7e14-47a4-88c5-ca621e9e43b1 None admin] RBAC: Adding query filter params (type=image) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:50.062307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5a27a-7e14-47a4-88c5-ca621e9e43b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:50.063670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5a27a-7e14-47a4-88c5-ca621e9e43b1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:50.069079 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 494/994] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:50.130498 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd73bd9-a47a-413e-bd18-06ca0245481c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:50.152008 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5a27a-7e14-47a4-88c5-ca621e9e43b1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:50.155407 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:50.196302 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c51a4-6ece-4a03-85d3-33f5ce50191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:50.197167 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c51a4-6ece-4a03-85d3-33f5ce50191a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:50.197666 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c51a4-6ece-4a03-85d3-33f5ce50191a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:50.197967 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c51a4-6ece-4a03-85d3-33f5ce50191a None admin] PATH_INFO: `/v3/endpoints` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:50.199364 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c51a4-6ece-4a03-85d3-33f5ce50191a None admin] RBAC: Adding query filter params (interface=admin, service_id=58130b126a8a429687f911a532cee176, region_id=RegionOne) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:50.199913 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c51a4-6ece-4a03-85d3-33f5ce50191a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:50.202580 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c51a4-6ece-4a03-85d3-33f5ce50191a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:50.212794 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 501/995] 10.222.0.33 () {56 vars in 1249 bytes} [Tue Oct 4 12:36:49 2022] GET /identity/v3/endpoints?service_id=58130b126a8a429687f911a532cee176&interface=admin®ion_id=RegionOne => generated 193 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:50.220003 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c51a4-6ece-4a03-85d3-33f5ce50191a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:50.275352 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:50.302684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5a27a-7e14-47a4-88c5-ca621e9e43b1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:50.350630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c69b4-fc83-4982-962a-757d4cf88fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:50.351269 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c69b4-fc83-4982-962a-757d4cf88fb9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:50.351587 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c69b4-fc83-4982-962a-757d4cf88fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:50.351861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c69b4-fc83-4982-962a-757d4cf88fb9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:50.352462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c69b4-fc83-4982-962a-757d4cf88fb9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:50.353847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c69b4-fc83-4982-962a-757d4cf88fb9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-8c9c69b4-fc83-4982-962a-757d4cf88fb9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:50.363405 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:50.364456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:50.365542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 495/996] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:50.369949 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c51a4-6ece-4a03-85d3-33f5ce50191a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:50.371508 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c69b4-fc83-4982-962a-757d4cf88fb9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:50.411570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f25fa7-c9fe-4ba1-987f-49118635223c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:50.412206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f25fa7-c9fe-4ba1-987f-49118635223c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:50.412510 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f25fa7-c9fe-4ba1-987f-49118635223c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:50.412764 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f25fa7-c9fe-4ba1-987f-49118635223c None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:50.413373 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f25fa7-c9fe-4ba1-987f-49118635223c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:50.414957 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f25fa7-c9fe-4ba1-987f-49118635223c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:50.422244 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 502/997] 10.222.0.33 () {56 vars in 1087 bytes} [Tue Oct 4 12:36:50 2022] GET /identity/v3/services => generated 1727 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:36:50.450851 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:50.492857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c69b4-fc83-4982-962a-757d4cf88fb9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:50.536310 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a8dd2-db2a-468a-bd85-f382228ca972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:50.536989 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a8dd2-db2a-468a-bd85-f382228ca972 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:50.537136 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a8dd2-db2a-468a-bd85-f382228ca972 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:50.537472 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a8dd2-db2a-468a-bd85-f382228ca972 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:50.538062 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a8dd2-db2a-468a-bd85-f382228ca972 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:50.538383 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a8dd2-db2a-468a-bd85-f382228ca972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:50.540379 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a8dd2-db2a-468a-bd85-f382228ca972 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:50.551115 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 496/998] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:50.553084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f25fa7-c9fe-4ba1-987f-49118635223c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:50.678103 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:50.682110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f25fa7-c9fe-4ba1-987f-49118635223c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:50.720377 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845504f-1804-4fde-abf8-f2485fae11b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:50.721046 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845504f-1804-4fde-abf8-f2485fae11b4 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:50.721323 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845504f-1804-4fde-abf8-f2485fae11b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:50.721539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845504f-1804-4fde-abf8-f2485fae11b4 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:50.722222 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845504f-1804-4fde-abf8-f2485fae11b4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:50.722524 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845504f-1804-4fde-abf8-f2485fae11b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:50.723926 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3845504f-1804-4fde-abf8-f2485fae11b4 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:50.730403 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 503/999] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:50.736515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a8dd2-db2a-468a-bd85-f382228ca972 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:50.869412 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a8dd2-db2a-468a-bd85-f382228ca972 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:50.920784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:50.931091 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba1623-0501-4786-a0ad-5b3f54796038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:50.932083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba1623-0501-4786-a0ad-5b3f54796038 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:50.932569 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba1623-0501-4786-a0ad-5b3f54796038 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:50.932898 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba1623-0501-4786-a0ad-5b3f54796038 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:50.934546 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba1623-0501-4786-a0ad-5b3f54796038 None admin] RBAC: Adding query filter params (interface=public, service_id=b2bbbb5e68b64096999fc813f42e23a1, region_id=RegionOne) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:50.935109 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba1623-0501-4786-a0ad-5b3f54796038 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:50.938256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba1623-0501-4786-a0ad-5b3f54796038 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:50.948592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 497/1000] 10.222.0.33 () {56 vars in 1251 bytes} [Tue Oct 4 12:36:50 2022] GET /identity/v3/endpoints?service_id=b2bbbb5e68b64096999fc813f42e23a1&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:50.956855 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845504f-1804-4fde-abf8-f2485fae11b4 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:51.133768 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845504f-1804-4fde-abf8-f2485fae11b4 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:51.200272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c3c35-8d24-48ce-a03f-8915e801a1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:51.201320 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3c35-8d24-48ce-a03f-8915e801a1f1 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:51.201805 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3c35-8d24-48ce-a03f-8915e801a1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:51.202337 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3c35-8d24-48ce-a03f-8915e801a1f1 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:51.203447 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c3c35-8d24-48ce-a03f-8915e801a1f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:51.206116 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c3c35-8d24-48ce-a03f-8915e801a1f1 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:51.219648 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 504/1001] 10.222.0.33 () {56 vars in 1087 bytes} [Tue Oct 4 12:36:50 2022] GET /identity/v3/services => generated 1727 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:36:52.351696 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c55c87-9a33-4810-acfa-01a874cf2a0b None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:52.352242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c55c87-9a33-4810-acfa-01a874cf2a0b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:52.352242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c55c87-9a33-4810-acfa-01a874cf2a0b None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:52.353876 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 498/1002] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:52.359614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3ce77-85af-4fa3-b04e-4225122eb931 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:52.360246 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3ce77-85af-4fa3-b04e-4225122eb931 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:52.360246 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3ce77-85af-4fa3-b04e-4225122eb931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:52.380084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-49f3ce77-85af-4fa3-b04e-4225122eb931 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:52.381358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-49f3ce77-85af-4fa3-b04e-4225122eb931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:52.419559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:52.484016 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 505/1003] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:52.744982 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119e17a-25f3-48ef-9763-4f3c0e94c99b None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:52.745326 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119e17a-25f3-48ef-9763-4f3c0e94c99b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:52.745326 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119e17a-25f3-48ef-9763-4f3c0e94c99b None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:52.746725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 499/1004] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:52.752722 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01594af0-773e-4aff-a0c0-3cabc2c05c1e None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:52.752932 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119e17a-25f3-48ef-9763-4f3c0e94c99b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:52.753023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01594af0-773e-4aff-a0c0-3cabc2c05c1e None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:52.753302 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01594af0-773e-4aff-a0c0-3cabc2c05c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:52.775636 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-01594af0-773e-4aff-a0c0-3cabc2c05c1e None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:52.776844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-01594af0-773e-4aff-a0c0-3cabc2c05c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:52.861653 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 506/1005] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:52.904965 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119e17a-25f3-48ef-9763-4f3c0e94c99b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:52.964564 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0457049-2eec-49c2-84f3-66f5cfd878d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:52.965331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0457049-2eec-49c2-84f3-66f5cfd878d6 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:52.965631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0457049-2eec-49c2-84f3-66f5cfd878d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:52.966000 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0457049-2eec-49c2-84f3-66f5cfd878d6 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:52.966827 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0457049-2eec-49c2-84f3-66f5cfd878d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:52.968457 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0457049-2eec-49c2-84f3-66f5cfd878d6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:52.987409 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 500/1006] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:36:52 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:36:53.137703 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01594af0-773e-4aff-a0c0-3cabc2c05c1e None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:53.263303 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:53.270031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01594af0-773e-4aff-a0c0-3cabc2c05c1e None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:53.312279 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa5e1c-d3c3-4064-ab71-78f2ec229a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:53.312971 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa5e1c-d3c3-4064-ab71-78f2ec229a19 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:53.313248 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa5e1c-d3c3-4064-ab71-78f2ec229a19 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:53.313513 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa5e1c-d3c3-4064-ab71-78f2ec229a19 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:53.314115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa5e1c-d3c3-4064-ab71-78f2ec229a19 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:53.315486 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa5e1c-d3c3-4064-ab71-78f2ec229a19 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-b0fa5e1c-d3c3-4064-ab71-78f2ec229a19 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:53.324684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:53.325836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:53.326829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 507/1007] 10.222.0.33 () {56 vars in 1126 bytes} [Tue Oct 4 12:36:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:53.331945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0457049-2eec-49c2-84f3-66f5cfd878d6 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:53.395784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:53.444289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0457049-2eec-49c2-84f3-66f5cfd878d6 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:53.479932 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd1442-d532-470a-a206-d522615e47f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:53.480544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd1442-d532-470a-a206-d522615e47f2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:53.480817 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd1442-d532-470a-a206-d522615e47f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:53.481082 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd1442-d532-470a-a206-d522615e47f2 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:53.481705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bd1442-d532-470a-a206-d522615e47f2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:53.481998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bd1442-d532-470a-a206-d522615e47f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:53.483382 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bd1442-d532-470a-a206-d522615e47f2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:53.488866 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 501/1008] 10.222.0.33 () {56 vars in 1122 bytes} [Tue Oct 4 12:36:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:53.491259 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbc6fd-b531-470b-b99e-9c2ad4e463c9 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:53.491745 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbc6fd-b531-470b-b99e-9c2ad4e463c9 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:53.491837 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbc6fd-b531-470b-b99e-9c2ad4e463c9 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:53.493304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 508/1009] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:53.494778 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd1442-d532-470a-a206-d522615e47f2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:53.498697 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de55d56-57ff-4a1a-99d4-cd541c58b5be None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:53.498981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de55d56-57ff-4a1a-99d4-cd541c58b5be None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:53.499252 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de55d56-57ff-4a1a-99d4-cd541c58b5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:53.519022 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-4de55d56-57ff-4a1a-99d4-cd541c58b5be None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:53.520221 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-4de55d56-57ff-4a1a-99d4-cd541c58b5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:53.592866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 509/1010] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:53 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:53.604355 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd1442-d532-470a-a206-d522615e47f2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:53.620141 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:53.639253 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39eab5-c26d-47d4-a7c5-9ba47fa23821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:53.639870 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39eab5-c26d-47d4-a7c5-9ba47fa23821 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:53.640145 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39eab5-c26d-47d4-a7c5-9ba47fa23821 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:53.640417 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39eab5-c26d-47d4-a7c5-9ba47fa23821 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:53.641046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39eab5-c26d-47d4-a7c5-9ba47fa23821 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:53.641360 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39eab5-c26d-47d4-a7c5-9ba47fa23821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:53.642763 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39eab5-c26d-47d4-a7c5-9ba47fa23821 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:53.653295 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 502/1011] 10.222.0.33 () {56 vars in 1122 bytes} [Tue Oct 4 12:36:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:53.655112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de55d56-57ff-4a1a-99d4-cd541c58b5be None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:53.746200 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:53.770315 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de55d56-57ff-4a1a-99d4-cd541c58b5be None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:53.804868 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691efe4f-bb31-4dc9-8a92-1c50756ba956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:53.805494 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691efe4f-bb31-4dc9-8a92-1c50756ba956 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:53.805768 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691efe4f-bb31-4dc9-8a92-1c50756ba956 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:53.806033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691efe4f-bb31-4dc9-8a92-1c50756ba956 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:53.806722 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691efe4f-bb31-4dc9-8a92-1c50756ba956 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:53.808121 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691efe4f-bb31-4dc9-8a92-1c50756ba956 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:53.830087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 510/1012] 10.222.0.33 () {60 vars in 1144 bytes} [Tue Oct 4 12:36:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:36:53.870984 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39eab5-c26d-47d4-a7c5-9ba47fa23821 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:53.990635 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39eab5-c26d-47d4-a7c5-9ba47fa23821 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:54.036242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75a0ab-4fe1-4592-92f4-5df70e97519f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:54.037229 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75a0ab-4fe1-4592-92f4-5df70e97519f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:54.037503 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75a0ab-4fe1-4592-92f4-5df70e97519f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:54.037782 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75a0ab-4fe1-4592-92f4-5df70e97519f None admin] PATH_INFO: `/v3/services/compute` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:54.038406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75a0ab-4fe1-4592-92f4-5df70e97519f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:54.039755 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75a0ab-4fe1-4592-92f4-5df70e97519f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-3a75a0ab-4fe1-4592-92f4-5df70e97519f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:54.048863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:54.049956 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:54.050846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 503/1013] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:54.058452 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691efe4f-bb31-4dc9-8a92-1c50756ba956 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:54.168885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691efe4f-bb31-4dc9-8a92-1c50756ba956 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:54.204139 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf1a32-40e7-4dc9-9383-ecb62031e5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:54.204742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf1a32-40e7-4dc9-9383-ecb62031e5cc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:54.205031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf1a32-40e7-4dc9-9383-ecb62031e5cc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:54.205278 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf1a32-40e7-4dc9-9383-ecb62031e5cc None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:54.205901 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf1a32-40e7-4dc9-9383-ecb62031e5cc None admin] RBAC: Adding query filter params (name=compute) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:54.206228 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf1a32-40e7-4dc9-9383-ecb62031e5cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:54.207667 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf1a32-40e7-4dc9-9383-ecb62031e5cc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:54.217737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 511/1014] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:54.218706 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75a0ab-4fe1-4592-92f4-5df70e97519f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:54.269491 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:54.330810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75a0ab-4fe1-4592-92f4-5df70e97519f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:54.369616 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7a9cf4-a913-4ebb-b805-02f39bf25d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:54.370265 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a9cf4-a913-4ebb-b805-02f39bf25d5b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:54.370541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a9cf4-a913-4ebb-b805-02f39bf25d5b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:54.370812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a9cf4-a913-4ebb-b805-02f39bf25d5b None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:54.371444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a9cf4-a913-4ebb-b805-02f39bf25d5b None admin] RBAC: Adding query filter params (type=compute) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:54.371742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a9cf4-a913-4ebb-b805-02f39bf25d5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:54.373109 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7a9cf4-a913-4ebb-b805-02f39bf25d5b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:54.382168 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 504/1015] 10.222.0.33 () {56 vars in 1112 bytes} [Tue Oct 4 12:36:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:54.384460 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf1a32-40e7-4dc9-9383-ecb62031e5cc None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:54.387362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:54.498666 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf1a32-40e7-4dc9-9383-ecb62031e5cc None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:54.539922 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66415b9d-adad-447b-a4d0-cde6d980d667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:54.540422 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66415b9d-adad-447b-a4d0-cde6d980d667 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:54.540697 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66415b9d-adad-447b-a4d0-cde6d980d667 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:54.540972 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66415b9d-adad-447b-a4d0-cde6d980d667 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:54.541649 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66415b9d-adad-447b-a4d0-cde6d980d667 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:54.543060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66415b9d-adad-447b-a4d0-cde6d980d667 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:54.574074 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 512/1016] 10.222.0.33 () {60 vars in 1144 bytes} [Tue Oct 4 12:36:54 2022] POST /identity/v3/endpoints => generated 334 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:36:55.233339 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c274a0-5811-41f2-9502-f474599f9462 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:55.234153 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c274a0-5811-41f2-9502-f474599f9462 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:55.234153 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c274a0-5811-41f2-9502-f474599f9462 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:55.235232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 505/1017] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:55.241103 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18fcc7-28a6-4a6b-bc87-f1199c19f4ba None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:55.241384 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18fcc7-28a6-4a6b-bc87-f1199c19f4ba None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:55.241659 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18fcc7-28a6-4a6b-bc87-f1199c19f4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:55.264753 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-0e18fcc7-28a6-4a6b-bc87-f1199c19f4ba None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:55.266307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-0e18fcc7-28a6-4a6b-bc87-f1199c19f4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:55.356469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:55.398314 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 513/1018] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:55.661791 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c274a0-5811-41f2-9502-f474599f9462 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:55.834817 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c274a0-5811-41f2-9502-f474599f9462 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:55.847884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:55.897456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287dadf-c0a8-4d7c-ba5a-a807c05758a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:55.898592 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287dadf-c0a8-4d7c-ba5a-a807c05758a4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:55.899078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287dadf-c0a8-4d7c-ba5a-a807c05758a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:55.899552 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287dadf-c0a8-4d7c-ba5a-a807c05758a4 None admin] PATH_INFO: `/v3/services/image` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:55.900603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287dadf-c0a8-4d7c-ba5a-a807c05758a4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:55.903244 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287dadf-c0a8-4d7c-ba5a-a807c05758a4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0287dadf-c0a8-4d7c-ba5a-a807c05758a4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:55.920387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:55.921604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:55.922931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 506/1019] 10.222.0.33 () {56 vars in 1105 bytes} [Tue Oct 4 12:36:55 2022] GET /identity/v3/services/image => generated 86 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:55.930864 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18fcc7-28a6-4a6b-bc87-f1199c19f4ba None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:56.072888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18fcc7-28a6-4a6b-bc87-f1199c19f4ba None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:56.074921 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a22917-3cc7-4af9-b335-81498d7d2446 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:56.075205 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a22917-3cc7-4af9-b335-81498d7d2446 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:56.075541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a22917-3cc7-4af9-b335-81498d7d2446 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:56.076824 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 507/1020] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:36:56.082446 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce8c25-b64d-4abc-a919-84ee40a8b8c8 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:56.082727 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce8c25-b64d-4abc-a919-84ee40a8b8c8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:56.083000 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce8c25-b64d-4abc-a919-84ee40a8b8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:56.106663 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-79ce8c25-b64d-4abc-a919-84ee40a8b8c8 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:56.107875 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-79ce8c25-b64d-4abc-a919-84ee40a8b8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:56.138349 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6415056-df39-4830-b7ff-af6a04b1786c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:56.139559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6415056-df39-4830-b7ff-af6a04b1786c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:56.140062 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6415056-df39-4830-b7ff-af6a04b1786c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:56.140549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6415056-df39-4830-b7ff-af6a04b1786c None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:56.141872 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6415056-df39-4830-b7ff-af6a04b1786c None admin] RBAC: Adding query filter params (name=image) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:56.142457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6415056-df39-4830-b7ff-af6a04b1786c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:56.145308 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6415056-df39-4830-b7ff-af6a04b1786c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:56.159218 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 514/1021] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:56.160648 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6415056-df39-4830-b7ff-af6a04b1786c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:56.202139 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 508/1022] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:56.226285 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:56.295680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6415056-df39-4830-b7ff-af6a04b1786c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:56.340057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55091824-6ca2-41a9-98ce-3b50f3c40206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:56.340682 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55091824-6ca2-41a9-98ce-3b50f3c40206 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:56.341277 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55091824-6ca2-41a9-98ce-3b50f3c40206 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:56.341603 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55091824-6ca2-41a9-98ce-3b50f3c40206 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:56.342372 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55091824-6ca2-41a9-98ce-3b50f3c40206 None admin] RBAC: Adding query filter params (type=image) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:56.342723 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55091824-6ca2-41a9-98ce-3b50f3c40206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:56.344088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55091824-6ca2-41a9-98ce-3b50f3c40206 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:56.350390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 515/1023] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:56.356630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce8c25-b64d-4abc-a919-84ee40a8b8c8 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:56.480433 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:56.484583 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55091824-6ca2-41a9-98ce-3b50f3c40206 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:56.490137 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce8c25-b64d-4abc-a919-84ee40a8b8c8 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:56.540219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c3854-4ae3-4fc6-9829-1709cf1b4436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:56.541306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c3854-4ae3-4fc6-9829-1709cf1b4436 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:56.541577 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c3854-4ae3-4fc6-9829-1709cf1b4436 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:56.541867 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c3854-4ae3-4fc6-9829-1709cf1b4436 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:56.542991 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c3854-4ae3-4fc6-9829-1709cf1b4436 None admin] RBAC: Adding query filter params (interface=public, service_id=ae1f098d9db446dc90a253e328b180e8, region_id=RegionOne) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:56.543462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c3854-4ae3-4fc6-9829-1709cf1b4436 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:56.544979 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c3854-4ae3-4fc6-9829-1709cf1b4436 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:56.552232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 509/1024] 10.222.0.33 () {56 vars in 1251 bytes} [Tue Oct 4 12:36:56 2022] GET /identity/v3/endpoints?service_id=ae1f098d9db446dc90a253e328b180e8&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:56.559446 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c3854-4ae3-4fc6-9829-1709cf1b4436 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:56.631507 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55091824-6ca2-41a9-98ce-3b50f3c40206 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:56.645398 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:56.670210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57b6b2-c4d7-4288-9083-b00c83380cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:56.670859 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57b6b2-c4d7-4288-9083-b00c83380cbf None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:56.671138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57b6b2-c4d7-4288-9083-b00c83380cbf None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:56.671411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57b6b2-c4d7-4288-9083-b00c83380cbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:56.672567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57b6b2-c4d7-4288-9083-b00c83380cbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:56.674031 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57b6b2-c4d7-4288-9083-b00c83380cbf None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:56.675673 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 516/1025] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:56.681947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c3854-4ae3-4fc6-9829-1709cf1b4436 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:56.682637 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57b6b2-c4d7-4288-9083-b00c83380cbf None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:56.727257 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4610a-da20-4044-aefa-8a9536f2bb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:56.728140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4610a-da20-4044-aefa-8a9536f2bb4e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:56.729171 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4610a-da20-4044-aefa-8a9536f2bb4e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:56.729603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4610a-da20-4044-aefa-8a9536f2bb4e None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:56.730495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4610a-da20-4044-aefa-8a9536f2bb4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:56.732551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4610a-da20-4044-aefa-8a9536f2bb4e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:56.741677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 510/1026] 10.222.0.33 () {56 vars in 1087 bytes} [Tue Oct 4 12:36:56 2022] GET /identity/v3/services => generated 1957 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:36:56.810308 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57b6b2-c4d7-4288-9083-b00c83380cbf None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:56.854658 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14238c01-b99d-4753-90d6-84473f9cf9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:56.855283 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14238c01-b99d-4753-90d6-84473f9cf9db None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:56.855562 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14238c01-b99d-4753-90d6-84473f9cf9db None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:56.855950 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14238c01-b99d-4753-90d6-84473f9cf9db None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:56.856483 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14238c01-b99d-4753-90d6-84473f9cf9db None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:56.857974 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14238c01-b99d-4753-90d6-84473f9cf9db None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:56.891541 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 517/1027] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:36:56 2022] POST /identity/v3/projects => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:36:56.904398 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144c613-59cc-43b4-9115-72ef337fb610 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:56.904912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144c613-59cc-43b4-9115-72ef337fb610 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:56.904912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144c613-59cc-43b4-9115-72ef337fb610 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:56.906725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 511/1028] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:56.912790 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158fc8a-e75d-4f25-bef8-1235feae7f6f None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:56.913181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158fc8a-e75d-4f25-bef8-1235feae7f6f None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:56.913332 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158fc8a-e75d-4f25-bef8-1235feae7f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:56.934299 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-3158fc8a-e75d-4f25-bef8-1235feae7f6f None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:56.935553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-3158fc8a-e75d-4f25-bef8-1235feae7f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:56.970819 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:57.027546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 518/1029] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:36:57.303547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144c613-59cc-43b4-9115-72ef337fb610 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:57.340466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:57.440384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144c613-59cc-43b4-9115-72ef337fb610 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:57.479169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299cb0d-2229-4514-9379-563cef209ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:57.479815 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299cb0d-2229-4514-9379-563cef209ebe None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:57.480096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299cb0d-2229-4514-9379-563cef209ebe None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:57.480358 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299cb0d-2229-4514-9379-563cef209ebe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:57.492728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299cb0d-2229-4514-9379-563cef209ebe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-d299cb0d-2229-4514-9379-563cef209ebe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:36:57.497540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:57.498842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:57.499914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 512/1030] 10.222.0.33 () {56 vars in 1120 bytes} [Tue Oct 4 12:36:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:57.507491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158fc8a-e75d-4f25-bef8-1235feae7f6f None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:57.678314 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158fc8a-e75d-4f25-bef8-1235feae7f6f None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:57.739421 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88ec7a-4d75-4def-8d58-c0f3e11697bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:57.739917 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88ec7a-4d75-4def-8d58-c0f3e11697bb None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:57.740196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88ec7a-4d75-4def-8d58-c0f3e11697bb None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:57.740468 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88ec7a-4d75-4def-8d58-c0f3e11697bb None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:57.741355 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88ec7a-4d75-4def-8d58-c0f3e11697bb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:57.741485 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88ec7a-4d75-4def-8d58-c0f3e11697bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:57.743431 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88ec7a-4d75-4def-8d58-c0f3e11697bb None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:57.762685 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 519/1031] 10.222.0.33 () {56 vars in 1115 bytes} [Tue Oct 4 12:36:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:57.773338 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299cb0d-2229-4514-9379-563cef209ebe None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:57.924378 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:57.939770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299cb0d-2229-4514-9379-563cef209ebe None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:57.977473 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc14b7-7584-4aca-9ecd-4a8ae0d4426d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:57.978219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc14b7-7584-4aca-9ecd-4a8ae0d4426d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:57.978631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc14b7-7584-4aca-9ecd-4a8ae0d4426d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:57.978960 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc14b7-7584-4aca-9ecd-4a8ae0d4426d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:57.990855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc14b7-7584-4aca-9ecd-4a8ae0d4426d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:57.992964 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc14b7-7584-4aca-9ecd-4a8ae0d4426d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0bdc14b7-7584-4aca-9ecd-4a8ae0d4426d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:58.008578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:58.009655 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:58.010582 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:58.010582 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:58.010582 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:58.010582 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 513/1032] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:58.017812 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88ec7a-4d75-4def-8d58-c0f3e11697bb None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:58.052713 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:58.152950 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88ec7a-4d75-4def-8d58-c0f3e11697bb None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:58.195956 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a2d68-2e78-468e-8012-c42427e33ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:58.196497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a2d68-2e78-468e-8012-c42427e33ee9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:58.196770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a2d68-2e78-468e-8012-c42427e33ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:58.197036 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a2d68-2e78-468e-8012-c42427e33ee9 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:58.197679 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a2d68-2e78-468e-8012-c42427e33ee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:58.197979 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a2d68-2e78-468e-8012-c42427e33ee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:58.199408 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a2d68-2e78-468e-8012-c42427e33ee9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:58.212413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 520/1033] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:58.218488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc14b7-7584-4aca-9ecd-4a8ae0d4426d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:58.377598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc14b7-7584-4aca-9ecd-4a8ae0d4426d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:58.435057 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865610-b9c8-4cb1-a36d-39a397cf9a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:58.435699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865610-b9c8-4cb1-a36d-39a397cf9a37 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:58.435968 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865610-b9c8-4cb1-a36d-39a397cf9a37 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:58.436234 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b865610-b9c8-4cb1-a36d-39a397cf9a37 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:58.444521 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865610-b9c8-4cb1-a36d-39a397cf9a37 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:58.446241 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b865610-b9c8-4cb1-a36d-39a397cf9a37 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:58.449107 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-3b865610-b9c8-4cb1-a36d-39a397cf9a37 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:36:58.460206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:58.461878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:58.463127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:58.463127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:58.463127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:36:58.463127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:36:58.463127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:36:58.463127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:36:58.463127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:36:58.463127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:36:58.463127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:36:58.463127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:58.465081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 514/1034] 10.222.0.33 () {56 vars in 1093 bytes} [Tue Oct 4 12:36:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:58.470847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a2d68-2e78-468e-8012-c42427e33ee9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:58.569116 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:58.591200 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a2d68-2e78-468e-8012-c42427e33ee9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:58.634518 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7779a6-c27d-4833-a48f-428b3cd0b12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:58.635125 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7779a6-c27d-4833-a48f-428b3cd0b12a None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:58.635393 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7779a6-c27d-4833-a48f-428b3cd0b12a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:58.635667 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7779a6-c27d-4833-a48f-428b3cd0b12a None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:58.636417 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7779a6-c27d-4833-a48f-428b3cd0b12a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:58.636723 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7779a6-c27d-4833-a48f-428b3cd0b12a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:58.638185 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7779a6-c27d-4833-a48f-428b3cd0b12a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:58.672618 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 521/1035] 10.222.0.33 () {56 vars in 1133 bytes} [Tue Oct 4 12:36:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:58.673951 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865610-b9c8-4cb1-a36d-39a397cf9a37 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:58.800985 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b865610-b9c8-4cb1-a36d-39a397cf9a37 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:58.844725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10409ca7-43ed-4d65-ad64-5d3f9a18294c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:58.845522 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10409ca7-43ed-4d65-ad64-5d3f9a18294c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:58.845922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10409ca7-43ed-4d65-ad64-5d3f9a18294c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:58.846360 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10409ca7-43ed-4d65-ad64-5d3f9a18294c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:58.927088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:58.932486 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10409ca7-43ed-4d65-ad64-5d3f9a18294c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:58.934365 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10409ca7-43ed-4d65-ad64-5d3f9a18294c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:59.079489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 515/1036] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:36:58 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:36:59.084973 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7779a6-c27d-4833-a48f-428b3cd0b12a None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:59.122237 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c32a58-4c59-42c9-9f39-ebdeedc2b567 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:59.122570 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c32a58-4c59-42c9-9f39-ebdeedc2b567 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:59.122790 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c32a58-4c59-42c9-9f39-ebdeedc2b567 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:59.124611 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 516/1037] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:59.129878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac4983-592c-4bce-8e72-12d7c613e1b5 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:59.130189 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac4983-592c-4bce-8e72-12d7c613e1b5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:59.130449 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac4983-592c-4bce-8e72-12d7c613e1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:59.156800 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1fac4983-592c-4bce-8e72-12d7c613e1b5 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:59.158840 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1fac4983-592c-4bce-8e72-12d7c613e1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:59.197859 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7779a6-c27d-4833-a48f-428b3cd0b12a None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:59.234320 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092e9b2-863f-4963-a405-312097e11a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:59.234973 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092e9b2-863f-4963-a405-312097e11a5c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:59.235247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092e9b2-863f-4963-a405-312097e11a5c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:59.235516 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6092e9b2-863f-4963-a405-312097e11a5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:59.236145 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 517/1038] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:59.237442 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd061455-a755-48a6-8b03-09ccd15805d2 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:59.237725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd061455-a755-48a6-8b03-09ccd15805d2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:59.237995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd061455-a755-48a6-8b03-09ccd15805d2 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:59.239348 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 518/1039] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:36:59.245645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db604334-33e3-4835-a4ce-0225358d70b2 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:59.245984 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db604334-33e3-4835-a4ce-0225358d70b2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:59.246319 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db604334-33e3-4835-a4ce-0225358d70b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:59.254712 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092e9b2-863f-4963-a405-312097e11a5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:59.256213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6092e9b2-863f-4963-a405-312097e11a5c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:59.259284 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:59.270185 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-db604334-33e3-4835-a4ce-0225358d70b2 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:36:59.271418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-db604334-33e3-4835-a4ce-0225358d70b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-6092e9b2-863f-4963-a405-312097e11a5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:36:59.275887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:59.276900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:36:59.277801 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:36:59.277801 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:36:59.277801 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:59.277801 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 522/1040] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:59.282954 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092e9b2-863f-4963-a405-312097e11a5c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:59.300887 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:59.354722 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 519/1041] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:36:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:36:59.397764 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6092e9b2-863f-4963-a405-312097e11a5c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:59.453207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c121d0-9232-4d96-aa3e-41b7d19d573e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:59.453813 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c121d0-9232-4d96-aa3e-41b7d19d573e None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:59.454117 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c121d0-9232-4d96-aa3e-41b7d19d573e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:59.454439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c121d0-9232-4d96-aa3e-41b7d19d573e None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:59.455087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c121d0-9232-4d96-aa3e-41b7d19d573e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:59.455388 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c121d0-9232-4d96-aa3e-41b7d19d573e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:59.456777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c121d0-9232-4d96-aa3e-41b7d19d573e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:59.473286 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 523/1042] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:36:59.475121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db604334-33e3-4835-a4ce-0225358d70b2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:59.505874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c121d0-9232-4d96-aa3e-41b7d19d573e None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:59.565827 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:59.579556 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db604334-33e3-4835-a4ce-0225358d70b2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:59.613400 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c121d0-9232-4d96-aa3e-41b7d19d573e None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:59.615725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a001bcb-57dc-4786-b7c3-1a45c04c6ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:59.616361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a001bcb-57dc-4786-b7c3-1a45c04c6ac7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:59.616634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a001bcb-57dc-4786-b7c3-1a45c04c6ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:59.617465 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a001bcb-57dc-4786-b7c3-1a45c04c6ac7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:59.624571 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a001bcb-57dc-4786-b7c3-1a45c04c6ac7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:59.626011 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a001bcb-57dc-4786-b7c3-1a45c04c6ac7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-2a001bcb-57dc-4786-b7c3-1a45c04c6ac7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:36:59.637453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:59.638684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:59.639661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:36:59.639661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:36:59.639661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:36:59.639661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:36:59.639661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:36:59.639661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:36:59.639661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:36:59.639661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:36:59.639661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 520/1043] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:36:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:36:59.639661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a001bcb-57dc-4786-b7c3-1a45c04c6ac7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:59.654504 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c206b-3ac4-4c4e-ac2a-5b76080c41f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:59.655145 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c206b-3ac4-4c4e-ac2a-5b76080c41f0 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:59.655412 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c206b-3ac4-4c4e-ac2a-5b76080c41f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:59.655681 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c206b-3ac4-4c4e-ac2a-5b76080c41f0 None admin] PATH_INFO: `/v3/services/image` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:59.656262 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c206b-3ac4-4c4e-ac2a-5b76080c41f0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:59.657630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c206b-3ac4-4c4e-ac2a-5b76080c41f0 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:59.662277 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-748c206b-3ac4-4c4e-ac2a-5b76080c41f0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:36:59.666654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:36:59.667692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:36:59.669093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 524/1044] 10.222.0.33 () {56 vars in 1105 bytes} [Tue Oct 4 12:36:59 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:36:59.669890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c206b-3ac4-4c4e-ac2a-5b76080c41f0 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:59.744497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a001bcb-57dc-4786-b7c3-1a45c04c6ac7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:59.770339 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c206b-3ac4-4c4e-ac2a-5b76080c41f0 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:59.786681 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218db73e-24a4-43f5-a902-495da79217d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:59.787313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218db73e-24a4-43f5-a902-495da79217d8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:59.787541 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:59.787659 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218db73e-24a4-43f5-a902-495da79217d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:59.787832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218db73e-24a4-43f5-a902-495da79217d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:59.789470 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218db73e-24a4-43f5-a902-495da79217d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:59.791081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218db73e-24a4-43f5-a902-495da79217d8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:59.792649 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 521/1045] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:59.793404 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218db73e-24a4-43f5-a902-495da79217d8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:59.806599 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf5a43-21c5-4314-b7d4-62bcd63b6240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:59.807203 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf5a43-21c5-4314-b7d4-62bcd63b6240 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:59.807476 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf5a43-21c5-4314-b7d4-62bcd63b6240 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:59.807740 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf5a43-21c5-4314-b7d4-62bcd63b6240 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:59.808420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf5a43-21c5-4314-b7d4-62bcd63b6240 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:59.808718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf5a43-21c5-4314-b7d4-62bcd63b6240 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:59.810191 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf5a43-21c5-4314-b7d4-62bcd63b6240 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:59.822684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 525/1046] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:36:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:59.823451 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf5a43-21c5-4314-b7d4-62bcd63b6240 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:59.888640 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:36:59.904473 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218db73e-24a4-43f5-a902-495da79217d8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:59.929919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf5a43-21c5-4314-b7d4-62bcd63b6240 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:36:59.938267 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90073cf-873a-45e0-8e76-c3b090e837bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:59.938880 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90073cf-873a-45e0-8e76-c3b090e837bc None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:59.939159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90073cf-873a-45e0-8e76-c3b090e837bc None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:59.939428 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90073cf-873a-45e0-8e76-c3b090e837bc None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:59.940054 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90073cf-873a-45e0-8e76-c3b090e837bc None admin] RBAC: Adding query filter params (name=image) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:36:59.940351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90073cf-873a-45e0-8e76-c3b090e837bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:59.941722 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90073cf-873a-45e0-8e76-c3b090e837bc None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:59.947248 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 522/1047] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:36:59.948033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90073cf-873a-45e0-8e76-c3b090e837bc None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:36:59.969358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d18c1a-cef1-4024-9291-a91cae76cf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:36:59.969961 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d18c1a-cef1-4024-9291-a91cae76cf5f None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:36:59.970267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d18c1a-cef1-4024-9291-a91cae76cf5f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:36:59.970537 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d18c1a-cef1-4024-9291-a91cae76cf5f None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:36:59.971181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d18c1a-cef1-4024-9291-a91cae76cf5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:36:59.972603 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d18c1a-cef1-4024-9291-a91cae76cf5f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:36:59.996650 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 526/1048] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:36:59 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:36:59.997477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d18c1a-cef1-4024-9291-a91cae76cf5f None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:00.044832 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:00.062935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90073cf-873a-45e0-8e76-c3b090e837bc None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:00.103141 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2a787-f8ad-4d4a-84a8-11a3cb2059cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:00.103761 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2a787-f8ad-4d4a-84a8-11a3cb2059cd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:00.104035 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2a787-f8ad-4d4a-84a8-11a3cb2059cd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:00.104309 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2a787-f8ad-4d4a-84a8-11a3cb2059cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:00.105036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2a787-f8ad-4d4a-84a8-11a3cb2059cd None admin] RBAC: Adding query filter params (role.id=0945a2902fad45818ce54d62ab8ff3eb, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=02260480264b4f8f8a7e5cb647ac5d9e) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:00.105337 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2a787-f8ad-4d4a-84a8-11a3cb2059cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:00.106832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2a787-f8ad-4d4a-84a8-11a3cb2059cd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:00.113099 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 523/1049] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:36:59 2022] GET /identity/v3/role_assignments?user.id=02260480264b4f8f8a7e5cb647ac5d9e&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=0945a2902fad45818ce54d62ab8ff3eb => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:00.143755 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d18c1a-cef1-4024-9291-a91cae76cf5f None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:00.187158 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f787009-cfa0-4128-b7b6-3cac7ddff0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:00.187788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f787009-cfa0-4128-b7b6-3cac7ddff0f4 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:00.188066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f787009-cfa0-4128-b7b6-3cac7ddff0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:00.188340 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f787009-cfa0-4128-b7b6-3cac7ddff0f4 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:00.188987 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f787009-cfa0-4128-b7b6-3cac7ddff0f4 None admin] RBAC: Adding query filter params (type=image) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:00.189334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f787009-cfa0-4128-b7b6-3cac7ddff0f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:00.190755 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f787009-cfa0-4128-b7b6-3cac7ddff0f4 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:00.198633 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 527/1050] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:36:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:00.204222 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2a787-f8ad-4d4a-84a8-11a3cb2059cd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:00.227674 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:00.323957 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2a787-f8ad-4d4a-84a8-11a3cb2059cd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:00.357081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cbd95-8203-4db1-a980-48a54b53b995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:00.357688 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cbd95-8203-4db1-a980-48a54b53b995 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:00.357965 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cbd95-8203-4db1-a980-48a54b53b995 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:00.358242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cbd95-8203-4db1-a980-48a54b53b995 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:00.358908 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cbd95-8203-4db1-a980-48a54b53b995 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:00.360262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cbd95-8203-4db1-a980-48a54b53b995 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:00.384080 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 524/1051] 10.222.0.33 () {60 vars in 1144 bytes} [Tue Oct 4 12:37:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:37:02.253843 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d1360-2112-4247-ad6d-a05a51d86231 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:02.254469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d1360-2112-4247-ad6d-a05a51d86231 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:02.254469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d1360-2112-4247-ad6d-a05a51d86231 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:02.257713 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 528/1052] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:02.261954 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a29e34-76b7-40e8-91a6-51b038efef32 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:02.262379 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a29e34-76b7-40e8-91a6-51b038efef32 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:02.262551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a29e34-76b7-40e8-91a6-51b038efef32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:02.282124 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-61a29e34-76b7-40e8-91a6-51b038efef32 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:02.283323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-61a29e34-76b7-40e8-91a6-51b038efef32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:02.286183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d8ddf-8110-4efd-bb8e-cea96968227a None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:02.287334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d8ddf-8110-4efd-bb8e-cea96968227a None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:02.287334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d8ddf-8110-4efd-bb8e-cea96968227a None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:02.288213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 529/1053] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:37:02.293860 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057a773-c9fe-4cdd-ba54-88554868c6a8 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:02.294174 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057a773-c9fe-4cdd-ba54-88554868c6a8 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:02.294447 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057a773-c9fe-4cdd-ba54-88554868c6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:02.315554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-9057a773-c9fe-4cdd-ba54-88554868c6a8 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:02.317520 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-9057a773-c9fe-4cdd-ba54-88554868c6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:02.363968 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:02.378091 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 525/1054] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:37:02.404201 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 530/1055] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:37:02.571814 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b39f0-1b48-43e5-b007-614b8d768faa None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:02.572155 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b39f0-1b48-43e5-b007-614b8d768faa None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:02.572262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b39f0-1b48-43e5-b007-614b8d768faa None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:02.573759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 526/1056] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:02.579044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08beeb-9e0d-4189-970c-41f73eb33d06 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:02.579333 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08beeb-9e0d-4189-970c-41f73eb33d06 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:02.579605 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08beeb-9e0d-4189-970c-41f73eb33d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:02.598312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-8b08beeb-9e0d-4189-970c-41f73eb33d06 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:02.599503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-8b08beeb-9e0d-4189-970c-41f73eb33d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:02.623568 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:02.645692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b39f0-1b48-43e5-b007-614b8d768faa None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:02.677957 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 531/1057] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:02.678676 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08beeb-9e0d-4189-970c-41f73eb33d06 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:02.758101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b39f0-1b48-43e5-b007-614b8d768faa None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:02.795728 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc2674-827e-4b50-920b-7ab7725e5076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:02.796391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc2674-827e-4b50-920b-7ab7725e5076 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:02.796674 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc2674-827e-4b50-920b-7ab7725e5076 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:02.796959 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc2674-827e-4b50-920b-7ab7725e5076 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:02.808762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc2674-827e-4b50-920b-7ab7725e5076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-8adc2674-827e-4b50-920b-7ab7725e5076 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:37:02.813622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:02.815410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:37:02.816998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 527/1058] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:37:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:02.822146 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc2674-827e-4b50-920b-7ab7725e5076 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:02.844196 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08beeb-9e0d-4189-970c-41f73eb33d06 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:02.887373 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb64b40-697d-4c50-b23f-e4c779eb5195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:02.888234 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb64b40-697d-4c50-b23f-e4c779eb5195 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:02.888234 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb64b40-697d-4c50-b23f-e4c779eb5195 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:02.888613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb64b40-697d-4c50-b23f-e4c779eb5195 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:02.902975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb64b40-697d-4c50-b23f-e4c779eb5195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-3fb64b40-697d-4c50-b23f-e4c779eb5195 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:37:02.908178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:02.909206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:02.910131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:02.910131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:02.910131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:02.910131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:02.910131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:37:02.910131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:37:02.910131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:37:02.910131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:37:02.910131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:37:02.910131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:37:02.910131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 12:37:02.910131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:02.910781 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:02.910856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 532/1059] 10.222.0.33 () {56 vars in 1120 bytes} [Tue Oct 4 12:37:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:02.915689 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb64b40-697d-4c50-b23f-e4c779eb5195 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:02.937094 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:02.962442 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc2674-827e-4b50-920b-7ab7725e5076 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:03.001313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ca239-8732-4866-9acf-b3e2d8640dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:03.001928 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ca239-8732-4866-9acf-b3e2d8640dcd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:03.002230 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ca239-8732-4866-9acf-b3e2d8640dcd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:03.002503 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ca239-8732-4866-9acf-b3e2d8640dcd None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:03.003172 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ca239-8732-4866-9acf-b3e2d8640dcd None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:03.003503 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ca239-8732-4866-9acf-b3e2d8640dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:03.004830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ca239-8732-4866-9acf-b3e2d8640dcd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:03.014905 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 528/1060] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:37:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:37:03.021259 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ca239-8732-4866-9acf-b3e2d8640dcd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:03.032586 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb64b40-697d-4c50-b23f-e4c779eb5195 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:03.067567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69127940-d01a-481a-a9f8-dea9333ddcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:03.068165 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69127940-d01a-481a-a9f8-dea9333ddcdb None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:03.068436 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69127940-d01a-481a-a9f8-dea9333ddcdb None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:03.068700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69127940-d01a-481a-a9f8-dea9333ddcdb None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:03.069346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69127940-d01a-481a-a9f8-dea9333ddcdb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:03.069646 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69127940-d01a-481a-a9f8-dea9333ddcdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:03.071617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69127940-d01a-481a-a9f8-dea9333ddcdb None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:03.080245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 533/1061] 10.222.0.33 () {56 vars in 1115 bytes} [Tue Oct 4 12:37:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:37:03.080990 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69127940-d01a-481a-a9f8-dea9333ddcdb None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:03.129457 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ca239-8732-4866-9acf-b3e2d8640dcd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:03.149856 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:03.161319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:03.164739 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c3b1f-0a89-4148-b054-13b8441c621b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:03.165371 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c3b1f-0a89-4148-b054-13b8441c621b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:03.165647 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c3b1f-0a89-4148-b054-13b8441c621b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:03.165914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c3b1f-0a89-4148-b054-13b8441c621b None admin] PATH_INFO: `/v3/users/19be0309a26848fc8643e6fa0150c7cc` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:03.180060 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c3b1f-0a89-4148-b054-13b8441c621b None admin] RBAC: Authorizing `identity:get_user(user_id=19be0309a26848fc8643e6fa0150c7cc)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:03.181375 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69127940-d01a-481a-a9f8-dea9333ddcdb None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:03.181496 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c3b1f-0a89-4148-b054-13b8441c621b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:03.183213 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 529/1062] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:03 2022] GET /identity/v3/users/19be0309a26848fc8643e6fa0150c7cc => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:37:03.183976 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c3b1f-0a89-4148-b054-13b8441c621b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:03.220812 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c56395-6330-4433-9f9b-87a2346035c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:03.221428 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c56395-6330-4433-9f9b-87a2346035c6 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:03.221706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c56395-6330-4433-9f9b-87a2346035c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:03.221974 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c56395-6330-4433-9f9b-87a2346035c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:03.288199 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c3b1f-0a89-4148-b054-13b8441c621b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:03.291773 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c56395-6330-4433-9f9b-87a2346035c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:03.293236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c56395-6330-4433-9f9b-87a2346035c6 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:03.323320 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb05ffd-6558-4eb8-850d-d22c3788948e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:03.323952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb05ffd-6558-4eb8-850d-d22c3788948e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:03.324228 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb05ffd-6558-4eb8-850d-d22c3788948e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:03.324497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb05ffd-6558-4eb8-850d-d22c3788948e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:03.334551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb05ffd-6558-4eb8-850d-d22c3788948e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:03.336561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb05ffd-6558-4eb8-850d-d22c3788948e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-6eb05ffd-6558-4eb8-850d-d22c3788948e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:37:03.350848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:03.351774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:37:03.352742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:37:03.352742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:37:03.352742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:37:03.352742 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 530/1063] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:03.353201 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb05ffd-6558-4eb8-850d-d22c3788948e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:03.376631 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:03.383507 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:03.397581 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 534/1064] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:37:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:37:03.399024 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c56395-6330-4433-9f9b-87a2346035c6 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:03.456217 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb05ffd-6558-4eb8-850d-d22c3788948e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:03.491217 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75ecea-6451-4880-b65b-e5531f75fa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:03.491854 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75ecea-6451-4880-b65b-e5531f75fa88 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:03.492138 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75ecea-6451-4880-b65b-e5531f75fa88 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:03.492408 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75ecea-6451-4880-b65b-e5531f75fa88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:03.498614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c56395-6330-4433-9f9b-87a2346035c6 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:03.535257 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef64435-aef3-4ccf-a483-9e58a6bc8166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:03.535862 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef64435-aef3-4ccf-a483-9e58a6bc8166 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:03.536136 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef64435-aef3-4ccf-a483-9e58a6bc8166 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:03.536403 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef64435-aef3-4ccf-a483-9e58a6bc8166 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:03.537035 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef64435-aef3-4ccf-a483-9e58a6bc8166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:03.537334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef64435-aef3-4ccf-a483-9e58a6bc8166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:03.538761 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef64435-aef3-4ccf-a483-9e58a6bc8166 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:03.550503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 535/1065] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:03.551319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef64435-aef3-4ccf-a483-9e58a6bc8166 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:03.562187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75ecea-6451-4880-b65b-e5531f75fa88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:03.563677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75ecea-6451-4880-b65b-e5531f75fa88 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:03.592267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:03.592758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:03.651428 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef64435-aef3-4ccf-a483-9e58a6bc8166 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:03.671063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 531/1066] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:37:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:37:03.671063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75ecea-6451-4880-b65b-e5531f75fa88 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:03.689669 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb98c29-c5d2-4b7b-8ccd-3cdb0f9ef3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:03.690326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb98c29-c5d2-4b7b-8ccd-3cdb0f9ef3f0 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:03.690606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb98c29-c5d2-4b7b-8ccd-3cdb0f9ef3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:03.690876 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb98c29-c5d2-4b7b-8ccd-3cdb0f9ef3f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:03.701770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb98c29-c5d2-4b7b-8ccd-3cdb0f9ef3f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:03.703239 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb98c29-c5d2-4b7b-8ccd-3cdb0f9ef3f0 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-9fb98c29-c5d2-4b7b-8ccd-3cdb0f9ef3f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:37:03.717033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:03.722220 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:37:03.723267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:37:03.723267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:37:03.723267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:03.723267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 536/1067] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:03.723267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb98c29-c5d2-4b7b-8ccd-3cdb0f9ef3f0 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:03.779389 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75ecea-6451-4880-b65b-e5531f75fa88 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:03.807825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:03.813391 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:03.814207 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0158529-9fd3-4563-93a1-380abc984d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:03.814839 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0158529-9fd3-4563-93a1-380abc984d6f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:03.815124 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0158529-9fd3-4563-93a1-380abc984d6f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:03.815391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0158529-9fd3-4563-93a1-380abc984d6f None admin] PATH_INFO: `/v3/users/nova` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:03.822770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb98c29-c5d2-4b7b-8ccd-3cdb0f9ef3f0 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:03.823003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0158529-9fd3-4563-93a1-380abc984d6f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:03.824448 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0158529-9fd3-4563-93a1-380abc984d6f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c0158529-9fd3-4563-93a1-380abc984d6f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:37:03.835914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:03.836941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:03.837892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:03.837892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:03.837892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:37:03.837892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:37:03.837892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:37:03.837892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:37:03.837892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:37:03.837892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:37:03.837892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:37:03.837892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:37:03.837892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 532/1068] 10.222.0.33 () {56 vars in 1093 bytes} [Tue Oct 4 12:37:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:03.837892 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0158529-9fd3-4563-93a1-380abc984d6f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:03.859154 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55cc3b-2a2a-47aa-a02d-9f3a8a0cc559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:03.859699 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55cc3b-2a2a-47aa-a02d-9f3a8a0cc559 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:03.859974 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55cc3b-2a2a-47aa-a02d-9f3a8a0cc559 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:03.860237 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55cc3b-2a2a-47aa-a02d-9f3a8a0cc559 None admin] PATH_INFO: `/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:03.861339 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55cc3b-2a2a-47aa-a02d-9f3a8a0cc559 None admin] RBAC: Authorizing `identity:get_project(project_id=b8bbf8ceacf24390a9bcf23ca62622f7)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:03.862951 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55cc3b-2a2a-47aa-a02d-9f3a8a0cc559 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:03.864534 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 537/1069] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:37:03 2022] GET /identity/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:03.865291 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55cc3b-2a2a-47aa-a02d-9f3a8a0cc559 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:03.940513 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0158529-9fd3-4563-93a1-380abc984d6f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:03.974524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb49c8-fb90-4e0b-b6f1-e87cbaa3e090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:03.974779 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55cc3b-2a2a-47aa-a02d-9f3a8a0cc559 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:03.975140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb49c8-fb90-4e0b-b6f1-e87cbaa3e090 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:03.975413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb49c8-fb90-4e0b-b6f1-e87cbaa3e090 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:03.975682 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb49c8-fb90-4e0b-b6f1-e87cbaa3e090 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:03.976857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb49c8-fb90-4e0b-b6f1-e87cbaa3e090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:03.977154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb49c8-fb90-4e0b-b6f1-e87cbaa3e090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:03.978546 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb49c8-fb90-4e0b-b6f1-e87cbaa3e090 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:03.990543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 533/1070] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:03.991359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb49c8-fb90-4e0b-b6f1-e87cbaa3e090 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:04.011312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea8eab-270a-4a14-98e4-9154b11e7c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:04.011945 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea8eab-270a-4a14-98e4-9154b11e7c94 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:04.012224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea8eab-270a-4a14-98e4-9154b11e7c94 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:04.012564 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ea8eab-270a-4a14-98e4-9154b11e7c94 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:04.013246 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea8eab-270a-4a14-98e4-9154b11e7c94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:04.013555 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea8eab-270a-4a14-98e4-9154b11e7c94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:04.015093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ea8eab-270a-4a14-98e4-9154b11e7c94 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:04.037890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 538/1071] 10.222.0.33 () {56 vars in 1133 bytes} [Tue Oct 4 12:37:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:04.044275 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea8eab-270a-4a14-98e4-9154b11e7c94 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:04.064525 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:04.064866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:04.128565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb49c8-fb90-4e0b-b6f1-e87cbaa3e090 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:04.162751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ea8eab-270a-4a14-98e4-9154b11e7c94 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:04.167039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cb8bb-7c17-4a04-89b0-0148aeb0fad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:04.167661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cb8bb-7c17-4a04-89b0-0148aeb0fad8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:04.167936 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cb8bb-7c17-4a04-89b0-0148aeb0fad8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:04.168204 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cb8bb-7c17-4a04-89b0-0148aeb0fad8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:04.168931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cb8bb-7c17-4a04-89b0-0148aeb0fad8 None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=b8bbf8ceacf24390a9bcf23ca62622f7, user.id=19be0309a26848fc8643e6fa0150c7cc) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:04.169239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cb8bb-7c17-4a04-89b0-0148aeb0fad8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:04.170712 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cb8bb-7c17-4a04-89b0-0148aeb0fad8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:04.177048 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 534/1072] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:37:03 2022] GET /identity/v3/role_assignments?user.id=19be0309a26848fc8643e6fa0150c7cc&scope.project.id=b8bbf8ceacf24390a9bcf23ca62622f7&role.id=fc68d072459d4720b9666c93bd535d63 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:04.206964 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd90f09-bdba-4c55-ab9c-a0e8140e4303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:04.207620 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd90f09-bdba-4c55-ab9c-a0e8140e4303 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:04.207894 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd90f09-bdba-4c55-ab9c-a0e8140e4303 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:04.208160 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd90f09-bdba-4c55-ab9c-a0e8140e4303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:04.220192 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd90f09-bdba-4c55-ab9c-a0e8140e4303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:04.221681 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd90f09-bdba-4c55-ab9c-a0e8140e4303 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-ffd90f09-bdba-4c55-ab9c-a0e8140e4303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:37:04.238442 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:04.239469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:37:04.240374 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:37:04.240374 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:37:04.240374 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:04.240374 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 539/1073] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:37:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:04.246819 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cb8bb-7c17-4a04-89b0-0148aeb0fad8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:04.371960 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cb8bb-7c17-4a04-89b0-0148aeb0fad8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:04.385590 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:04.407171 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78556b73-aed7-47b5-83a5-cb669b9f3233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:04.407785 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78556b73-aed7-47b5-83a5-cb669b9f3233 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:04.408061 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78556b73-aed7-47b5-83a5-cb669b9f3233 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:04.408338 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78556b73-aed7-47b5-83a5-cb669b9f3233 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:04.408976 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78556b73-aed7-47b5-83a5-cb669b9f3233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:04.409293 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78556b73-aed7-47b5-83a5-cb669b9f3233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:04.410713 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78556b73-aed7-47b5-83a5-cb669b9f3233 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:04.423637 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 535/1074] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:04.429093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd90f09-bdba-4c55-ab9c-a0e8140e4303 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:04.498161 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:04.530112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd90f09-bdba-4c55-ab9c-a0e8140e4303 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:04.564908 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbd89b-667d-4eab-a9e5-8093af76e36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:04.565523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbd89b-667d-4eab-a9e5-8093af76e36b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:04.565797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbd89b-667d-4eab-a9e5-8093af76e36b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:04.566092 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbd89b-667d-4eab-a9e5-8093af76e36b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:04.572661 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbd89b-667d-4eab-a9e5-8093af76e36b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:04.574124 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbd89b-667d-4eab-a9e5-8093af76e36b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-facbd89b-667d-4eab-a9e5-8093af76e36b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:37:04.585083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:04.586546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:04.587423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:37:04.587423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:37:04.587423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:37:04.587423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:37:04.587423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:37:04.587423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:37:04.587423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:37:04.587423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:04.587423 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 540/1075] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:37:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:04.592528 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78556b73-aed7-47b5-83a5-cb669b9f3233 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:04.690879 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78556b73-aed7-47b5-83a5-cb669b9f3233 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:04.724013 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dcc56-ef7c-4202-a254-c6de96b56226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:04.724619 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dcc56-ef7c-4202-a254-c6de96b56226 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:04.724894 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dcc56-ef7c-4202-a254-c6de96b56226 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:04.725164 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dcc56-ef7c-4202-a254-c6de96b56226 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:04.725863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291dcc56-ef7c-4202-a254-c6de96b56226 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:04.726726 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291dcc56-ef7c-4202-a254-c6de96b56226 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:04.728152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291dcc56-ef7c-4202-a254-c6de96b56226 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:04.739523 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 536/1076] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:37:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:04.745461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbd89b-667d-4eab-a9e5-8093af76e36b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:04.848588 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbd89b-667d-4eab-a9e5-8093af76e36b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:04.877409 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:04.883828 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60d6f1-de2b-412f-9048-644e489d9ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:04.884490 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60d6f1-de2b-412f-9048-644e489d9ab4 None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:04.884776 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60d6f1-de2b-412f-9048-644e489d9ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:04.885047 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60d6f1-de2b-412f-9048-644e489d9ab4 None admin] PATH_INFO: `/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/02260480264b4f8f8a7e5cb647ac5d9e/roles/0945a2902fad45818ce54d62ab8ff3eb` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:04.887304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60d6f1-de2b-412f-9048-644e489d9ab4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fb003a695490faa0077bbbe5e7d77, user_id=02260480264b4f8f8a7e5cb647ac5d9e, role_id=0945a2902fad45818ce54d62ab8ff3eb)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:04.888749 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60d6f1-de2b-412f-9048-644e489d9ab4 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:04.902467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 541/1077] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:37:04 2022] PUT /identity/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/02260480264b4f8f8a7e5cb647ac5d9e/roles/0945a2902fad45818ce54d62ab8ff3eb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:37:06.334195 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61148354-4a15-41a2-a04c-c86f16c37250 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:06.334712 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61148354-4a15-41a2-a04c-c86f16c37250 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:06.334801 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61148354-4a15-41a2-a04c-c86f16c37250 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:06.336672 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 537/1078] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:06.341957 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a42719-b492-40b8-a9ff-49609578ab66 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:06.342440 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a42719-b492-40b8-a9ff-49609578ab66 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:06.342630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a42719-b492-40b8-a9ff-49609578ab66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:06.360748 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-97a42719-b492-40b8-a9ff-49609578ab66 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:06.361950 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-97a42719-b492-40b8-a9ff-49609578ab66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:06.428874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 542/1079] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:06.697159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61148354-4a15-41a2-a04c-c86f16c37250 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:06.732886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:06.802237 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61148354-4a15-41a2-a04c-c86f16c37250 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:06.834484 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeb34a-f1b6-4dc0-81f6-8c3d19569c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:06.835105 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeb34a-f1b6-4dc0-81f6-8c3d19569c30 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:06.835904 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeb34a-f1b6-4dc0-81f6-8c3d19569c30 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:06.836170 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeb34a-f1b6-4dc0-81f6-8c3d19569c30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:06.845862 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eeb34a-f1b6-4dc0-81f6-8c3d19569c30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c8eeb34a-f1b6-4dc0-81f6-8c3d19569c30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:37:06.850540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:06.852307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:06.853352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:06.853352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:06.853352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:06.853352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:06.853352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:37:06.853352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:37:06.853352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:37:06.853352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:37:06.853352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:37:06.853352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:37:06.853352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:06.853352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:37:06.857156 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 538/1080] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:37:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:06.858109 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a42719-b492-40b8-a9ff-49609578ab66 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:06.959653 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a42719-b492-40b8-a9ff-49609578ab66 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:06.982515 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:06.997674 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a3234a-abb5-4ba1-8e32-53f090936342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:06.998311 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3234a-abb5-4ba1-8e32-53f090936342 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:06.998586 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3234a-abb5-4ba1-8e32-53f090936342 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:06.998853 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a3234a-abb5-4ba1-8e32-53f090936342 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:06.999516 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3234a-abb5-4ba1-8e32-53f090936342 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:06.999816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3234a-abb5-4ba1-8e32-53f090936342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:07.001179 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a3234a-abb5-4ba1-8e32-53f090936342 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:07.009793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 543/1081] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:37:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:07.015582 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeb34a-f1b6-4dc0-81f6-8c3d19569c30 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:07.063595 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584239e0-8296-4cca-9acb-25ae4511e6b9 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:07.063996 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584239e0-8296-4cca-9acb-25ae4511e6b9 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:07.064268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584239e0-8296-4cca-9acb-25ae4511e6b9 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:07.065600 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 544/1082] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:07.071212 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b95810-8106-451f-86c3-84a2eb7f06cf None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:07.071492 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b95810-8106-451f-86c3-84a2eb7f06cf None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:07.071792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b95810-8106-451f-86c3-84a2eb7f06cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:07.090174 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-88b95810-8106-451f-86c3-84a2eb7f06cf None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:07.091378 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-88b95810-8106-451f-86c3-84a2eb7f06cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:07.111914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:07.119612 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeb34a-f1b6-4dc0-81f6-8c3d19569c30 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:07.155084 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06ba44-f489-48ae-87b5-f4b80bde3df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:07.155735 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06ba44-f489-48ae-87b5-f4b80bde3df5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:07.156011 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06ba44-f489-48ae-87b5-f4b80bde3df5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:07.156278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06ba44-f489-48ae-87b5-f4b80bde3df5 None admin] PATH_INFO: `/v3/users/19be0309a26848fc8643e6fa0150c7cc` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:07.157456 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06ba44-f489-48ae-87b5-f4b80bde3df5 None admin] RBAC: Authorizing `identity:get_user(user_id=19be0309a26848fc8643e6fa0150c7cc)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:07.158092 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 545/1083] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:37:07.158959 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06ba44-f489-48ae-87b5-f4b80bde3df5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:07.165110 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 539/1084] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:07 2022] GET /identity/v3/users/19be0309a26848fc8643e6fa0150c7cc => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:07.166336 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b95810-8106-451f-86c3-84a2eb7f06cf None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:07.265604 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b95810-8106-451f-86c3-84a2eb7f06cf None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:07.279957 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:07.300327 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcb10d-fe18-48f1-b810-628365826f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:07.300941 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcb10d-fe18-48f1-b810-628365826f4c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:07.301213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcb10d-fe18-48f1-b810-628365826f4c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:07.301482 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcb10d-fe18-48f1-b810-628365826f4c None admin] PATH_INFO: `/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:07.302602 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcb10d-fe18-48f1-b810-628365826f4c None admin] RBAC: Authorizing `identity:get_project(project_id=b8bbf8ceacf24390a9bcf23ca62622f7)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:07.304023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcb10d-fe18-48f1-b810-628365826f4c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:07.305979 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 546/1085] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:37:07 2022] GET /identity/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:07.311447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06ba44-f489-48ae-87b5-f4b80bde3df5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:07.412569 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06ba44-f489-48ae-87b5-f4b80bde3df5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:07.426072 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcb10d-fe18-48f1-b810-628365826f4c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:07.447602 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067709e-486d-480b-b75a-667c96a02109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:07.448263 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067709e-486d-480b-b75a-667c96a02109 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:07.448537 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067709e-486d-480b-b75a-667c96a02109 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:07.448805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067709e-486d-480b-b75a-667c96a02109 None admin] PATH_INFO: `/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7/users/19be0309a26848fc8643e6fa0150c7cc/roles/fc68d072459d4720b9666c93bd535d63` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:07.450743 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067709e-486d-480b-b75a-667c96a02109 None admin] RBAC: Authorizing `identity:create_grant(project_id=b8bbf8ceacf24390a9bcf23ca62622f7, user_id=19be0309a26848fc8643e6fa0150c7cc, role_id=fc68d072459d4720b9666c93bd535d63)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:07.452173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067709e-486d-480b-b75a-667c96a02109 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:07.465820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 540/1086] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:37:07 2022] PUT /identity/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7/users/19be0309a26848fc8643e6fa0150c7cc/roles/fc68d072459d4720b9666c93bd535d63 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:37:07.533865 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcb10d-fe18-48f1-b810-628365826f4c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:07.571828 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc2cd0-ddf5-4643-bbb9-395927a1eef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:07.572476 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc2cd0-ddf5-4643-bbb9-395927a1eef2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:07.572752 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc2cd0-ddf5-4643-bbb9-395927a1eef2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:07.573037 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc2cd0-ddf5-4643-bbb9-395927a1eef2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:07.584383 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc2cd0-ddf5-4643-bbb9-395927a1eef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-15fc2cd0-ddf5-4643-bbb9-395927a1eef2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:37:07.589236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:07.590432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:07.591272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 547/1087] 10.222.0.33 () {56 vars in 1120 bytes} [Tue Oct 4 12:37:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:37:07.597068 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067709e-486d-480b-b75a-667c96a02109 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:07.624357 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:07.710490 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067709e-486d-480b-b75a-667c96a02109 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:07.745300 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6216aae-739f-42a1-ba14-840ecf61bc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:07.745912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6216aae-739f-42a1-ba14-840ecf61bc36 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:07.746230 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6216aae-739f-42a1-ba14-840ecf61bc36 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:07.746505 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6216aae-739f-42a1-ba14-840ecf61bc36 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:07.747165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6216aae-739f-42a1-ba14-840ecf61bc36 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:07.747468 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6216aae-739f-42a1-ba14-840ecf61bc36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:07.748832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6216aae-739f-42a1-ba14-840ecf61bc36 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:07.761532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 541/1088] 10.222.0.33 () {56 vars in 1115 bytes} [Tue Oct 4 12:37:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:07.762636 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc2cd0-ddf5-4643-bbb9-395927a1eef2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:07.806384 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:07.865065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc2cd0-ddf5-4643-bbb9-395927a1eef2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:07.900111 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6ed40-9704-4fd7-ab64-b002c1c7b061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:07.900740 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6ed40-9704-4fd7-ab64-b002c1c7b061 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:07.901023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6ed40-9704-4fd7-ab64-b002c1c7b061 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:07.901297 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6ed40-9704-4fd7-ab64-b002c1c7b061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:07.911338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6ed40-9704-4fd7-ab64-b002c1c7b061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:07.912792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6ed40-9704-4fd7-ab64-b002c1c7b061 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-7ed6ed40-9704-4fd7-ab64-b002c1c7b061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:37:07.927675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:07.928899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:37:07.929830 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:37:07.929830 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:37:07.929830 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:07.929830 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 548/1089] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:37:07.935069 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6216aae-739f-42a1-ba14-840ecf61bc36 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:08.024388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:08.040022 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6216aae-739f-42a1-ba14-840ecf61bc36 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:08.074110 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393cbb0-1fbd-4b5e-ad71-e4fcd10dd9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:08.074650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393cbb0-1fbd-4b5e-ad71-e4fcd10dd9f1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:08.074927 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393cbb0-1fbd-4b5e-ad71-e4fcd10dd9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:08.075199 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393cbb0-1fbd-4b5e-ad71-e4fcd10dd9f1 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:08.075837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393cbb0-1fbd-4b5e-ad71-e4fcd10dd9f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:08.076137 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393cbb0-1fbd-4b5e-ad71-e4fcd10dd9f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:08.077506 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393cbb0-1fbd-4b5e-ad71-e4fcd10dd9f1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:08.091084 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 542/1090] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:08.096885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6ed40-9704-4fd7-ab64-b002c1c7b061 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:08.189615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:08.199078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6ed40-9704-4fd7-ab64-b002c1c7b061 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:08.231680 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3ff76-fc95-4d9a-aec7-7175ed6e00b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:08.232294 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3ff76-fc95-4d9a-aec7-7175ed6e00b1 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:08.232567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3ff76-fc95-4d9a-aec7-7175ed6e00b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:08.232834 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3ff76-fc95-4d9a-aec7-7175ed6e00b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:08.240116 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3ff76-fc95-4d9a-aec7-7175ed6e00b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:08.242058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3ff76-fc95-4d9a-aec7-7175ed6e00b1 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-95d3ff76-fc95-4d9a-aec7-7175ed6e00b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:37:08.253058 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:08.254068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:08.255055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:08.255055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:08.255055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:37:08.255055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:37:08.255055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:37:08.255055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:37:08.255055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:37:08.255055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:37:08.255055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 12:37:08.255055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:08.255055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 549/1091] 10.222.0.33 () {56 vars in 1093 bytes} [Tue Oct 4 12:37:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:08.262569 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393cbb0-1fbd-4b5e-ad71-e4fcd10dd9f1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:08.362985 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393cbb0-1fbd-4b5e-ad71-e4fcd10dd9f1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:08.395817 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7542a1-dbfd-476f-b925-2931d8e828a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:08.396427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7542a1-dbfd-476f-b925-2931d8e828a7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:08.396697 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7542a1-dbfd-476f-b925-2931d8e828a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:08.396961 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7542a1-dbfd-476f-b925-2931d8e828a7 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:08.398205 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7542a1-dbfd-476f-b925-2931d8e828a7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:08.398503 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7542a1-dbfd-476f-b925-2931d8e828a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:08.399916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7542a1-dbfd-476f-b925-2931d8e828a7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:08.417611 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 543/1092] 10.222.0.33 () {56 vars in 1133 bytes} [Tue Oct 4 12:37:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:08.423495 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3ff76-fc95-4d9a-aec7-7175ed6e00b1 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:08.523877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3ff76-fc95-4d9a-aec7-7175ed6e00b1 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:08.557433 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80853416-13ac-4215-85a8-affa3799a5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:08.557986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80853416-13ac-4215-85a8-affa3799a5e8 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:08.558320 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80853416-13ac-4215-85a8-affa3799a5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:08.558578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80853416-13ac-4215-85a8-affa3799a5e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:08.580827 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:08.627768 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80853416-13ac-4215-85a8-affa3799a5e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:08.629114 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80853416-13ac-4215-85a8-affa3799a5e8 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:08.729007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 550/1093] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:37:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:37:08.735006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7542a1-dbfd-476f-b925-2931d8e828a7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:08.737957 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:08.836830 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7542a1-dbfd-476f-b925-2931d8e828a7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:08.871789 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460bb5d-b58a-4026-9456-ffcf35be59e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:08.872416 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460bb5d-b58a-4026-9456-ffcf35be59e1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:08.872687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460bb5d-b58a-4026-9456-ffcf35be59e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:08.872952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460bb5d-b58a-4026-9456-ffcf35be59e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:08.883975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460bb5d-b58a-4026-9456-ffcf35be59e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:08.885338 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460bb5d-b58a-4026-9456-ffcf35be59e1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a460bb5d-b58a-4026-9456-ffcf35be59e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:37:08.900020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:08.901284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:37:08.902508 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:37:08.902508 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:37:08.902508 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:37:08.905863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 544/1094] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:08.906742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80853416-13ac-4215-85a8-affa3799a5e8 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:08.958607 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:09.005632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80853416-13ac-4215-85a8-affa3799a5e8 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:09.040293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a7cac-8f61-46d6-9f4f-d743068a2a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:09.041016 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a7cac-8f61-46d6-9f4f-d743068a2a2d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:09.041287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a7cac-8f61-46d6-9f4f-d743068a2a2d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:09.041554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a7cac-8f61-46d6-9f4f-d743068a2a2d None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:09.042213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a7cac-8f61-46d6-9f4f-d743068a2a2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:09.042508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a7cac-8f61-46d6-9f4f-d743068a2a2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:09.043909 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a7cac-8f61-46d6-9f4f-d743068a2a2d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:09.056307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 551/1095] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:09.062420 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460bb5d-b58a-4026-9456-ffcf35be59e1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:09.120780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:09.168648 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460bb5d-b58a-4026-9456-ffcf35be59e1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:09.204059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b25352-441d-4d2a-830a-54ca5b804fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:09.204689 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b25352-441d-4d2a-830a-54ca5b804fa7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:09.204965 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b25352-441d-4d2a-830a-54ca5b804fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:09.205232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b25352-441d-4d2a-830a-54ca5b804fa7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:09.212559 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b25352-441d-4d2a-830a-54ca5b804fa7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:09.214008 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b25352-441d-4d2a-830a-54ca5b804fa7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-49b25352-441d-4d2a-830a-54ca5b804fa7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:37:09.225406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:09.227036 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:09.228634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:37:09.228634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:37:09.228634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:37:09.228634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:37:09.228634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:37:09.228634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:37:09.228634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:37:09.228634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:37:09.228634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 545/1096] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:37:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:09.233046 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a7cac-8f61-46d6-9f4f-d743068a2a2d None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:09.334309 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a7cac-8f61-46d6-9f4f-d743068a2a2d None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:09.347947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:09.371617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c365af2-e219-4a09-b7ac-807daa9a55dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:09.372247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c365af2-e219-4a09-b7ac-807daa9a55dc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:09.372521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c365af2-e219-4a09-b7ac-807daa9a55dc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:09.372787 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c365af2-e219-4a09-b7ac-807daa9a55dc None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:09.373492 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c365af2-e219-4a09-b7ac-807daa9a55dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:09.373794 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c365af2-e219-4a09-b7ac-807daa9a55dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:09.375312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c365af2-e219-4a09-b7ac-807daa9a55dc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:09.388727 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 552/1097] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:37:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:09.395343 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b25352-441d-4d2a-830a-54ca5b804fa7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:09.505050 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b25352-441d-4d2a-830a-54ca5b804fa7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:09.521578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:09.545212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa904495-0807-4467-bb83-4cc9a933c6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:09.545817 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa904495-0807-4467-bb83-4cc9a933c6a2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:09.546120 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa904495-0807-4467-bb83-4cc9a933c6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:09.546395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa904495-0807-4467-bb83-4cc9a933c6a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:09.547124 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa904495-0807-4467-bb83-4cc9a933c6a2 None admin] RBAC: Adding query filter params (role.id=0945a2902fad45818ce54d62ab8ff3eb, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=02260480264b4f8f8a7e5cb647ac5d9e) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:09.547431 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa904495-0807-4467-bb83-4cc9a933c6a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:09.548862 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa904495-0807-4467-bb83-4cc9a933c6a2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:09.555999 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 546/1098] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:37:09 2022] GET /identity/v3/role_assignments?user.id=02260480264b4f8f8a7e5cb647ac5d9e&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=0945a2902fad45818ce54d62ab8ff3eb => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:37:09.629391 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8298d7-ddcd-4195-9175-bb70af2209fc None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:09.629933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8298d7-ddcd-4195-9175-bb70af2209fc None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:09.629933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8298d7-ddcd-4195-9175-bb70af2209fc None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:09.631335 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 553/1099] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:09.637387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ca05e-1894-4665-b4d1-38b183a802be None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:09.637668 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ca05e-1894-4665-b4d1-38b183a802be None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:09.637943 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ca05e-1894-4665-b4d1-38b183a802be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:09.658015 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-780ca05e-1894-4665-b4d1-38b183a802be None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:09.659264 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-780ca05e-1894-4665-b4d1-38b183a802be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:09.732432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 547/1100] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:10.002028 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8298d7-ddcd-4195-9175-bb70af2209fc None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:10.110991 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8298d7-ddcd-4195-9175-bb70af2209fc None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:10.146607 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a000dd-b9a3-4987-8076-b1a8ecb388c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:10.147235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a000dd-b9a3-4987-8076-b1a8ecb388c7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:10.147511 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a000dd-b9a3-4987-8076-b1a8ecb388c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:10.147815 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a000dd-b9a3-4987-8076-b1a8ecb388c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:10.158263 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a000dd-b9a3-4987-8076-b1a8ecb388c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-42a000dd-b9a3-4987-8076-b1a8ecb388c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:37:10.163635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:10.164582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:10.165411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 554/1101] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:37:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:10.170822 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ca05e-1894-4665-b4d1-38b183a802be None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:10.277863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ca05e-1894-4665-b4d1-38b183a802be None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:10.289406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:10.318268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42dd3f-d773-4f46-8db5-5930314d3c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:10.318884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42dd3f-d773-4f46-8db5-5930314d3c1f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:10.319158 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42dd3f-d773-4f46-8db5-5930314d3c1f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:10.319427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42dd3f-d773-4f46-8db5-5930314d3c1f None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:10.320087 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42dd3f-d773-4f46-8db5-5930314d3c1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:10.320385 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42dd3f-d773-4f46-8db5-5930314d3c1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:10.321758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42dd3f-d773-4f46-8db5-5930314d3c1f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:10.330461 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 548/1102] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:37:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:10.336517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a000dd-b9a3-4987-8076-b1a8ecb388c7 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:10.368908 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:10.441104 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a000dd-b9a3-4987-8076-b1a8ecb388c7 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:10.475359 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245e4c3-5d56-4e39-bfb9-8cbc2b398df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:10.476630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245e4c3-5d56-4e39-bfb9-8cbc2b398df5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:10.476904 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245e4c3-5d56-4e39-bfb9-8cbc2b398df5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:10.477169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245e4c3-5d56-4e39-bfb9-8cbc2b398df5 None admin] PATH_INFO: `/v3/users/19be0309a26848fc8643e6fa0150c7cc` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:10.478307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245e4c3-5d56-4e39-bfb9-8cbc2b398df5 None admin] RBAC: Authorizing `identity:get_user(user_id=19be0309a26848fc8643e6fa0150c7cc)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:10.480787 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245e4c3-5d56-4e39-bfb9-8cbc2b398df5 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:10.482376 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 555/1103] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:10 2022] GET /identity/v3/users/19be0309a26848fc8643e6fa0150c7cc => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:10.488163 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42dd3f-d773-4f46-8db5-5930314d3c1f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:10.585508 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42dd3f-d773-4f46-8db5-5930314d3c1f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:10.617968 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a34e1-44e0-440f-86a4-8ccae24637bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:10.618614 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a34e1-44e0-440f-86a4-8ccae24637bd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:10.618890 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a34e1-44e0-440f-86a4-8ccae24637bd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:10.619161 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a34e1-44e0-440f-86a4-8ccae24637bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:10.657381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:10.686186 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a34e1-44e0-440f-86a4-8ccae24637bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:10.687640 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a34e1-44e0-440f-86a4-8ccae24637bd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:10.787381 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 549/1104] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:37:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:37:10.793378 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245e4c3-5d56-4e39-bfb9-8cbc2b398df5 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:10.894591 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:10.898344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245e4c3-5d56-4e39-bfb9-8cbc2b398df5 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:10.930968 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689b714-aa2a-427a-abe0-90d2cdd5761f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:10.931578 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689b714-aa2a-427a-abe0-90d2cdd5761f None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:10.931850 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689b714-aa2a-427a-abe0-90d2cdd5761f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:10.932118 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689b714-aa2a-427a-abe0-90d2cdd5761f None admin] PATH_INFO: `/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:10.933214 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689b714-aa2a-427a-abe0-90d2cdd5761f None admin] RBAC: Authorizing `identity:get_project(project_id=b8bbf8ceacf24390a9bcf23ca62622f7)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:10.934672 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689b714-aa2a-427a-abe0-90d2cdd5761f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:10.936258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 556/1105] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:37:10 2022] GET /identity/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:10.943451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a34e1-44e0-440f-86a4-8ccae24637bd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:11.012403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:11.043829 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a34e1-44e0-440f-86a4-8ccae24637bd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:11.076361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5def0-562a-468a-ab09-f47376e74751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:11.076958 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5def0-562a-468a-ab09-f47376e74751 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:11.077227 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5def0-562a-468a-ab09-f47376e74751 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:11.077490 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5def0-562a-468a-ab09-f47376e74751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:11.078228 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5def0-562a-468a-ab09-f47376e74751 None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=b8bbf8ceacf24390a9bcf23ca62622f7, user.id=19be0309a26848fc8643e6fa0150c7cc) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:11.078532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5def0-562a-468a-ab09-f47376e74751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:11.079940 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5def0-562a-468a-ab09-f47376e74751 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:11.086096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 550/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:37:10 2022] GET /identity/v3/role_assignments?user.id=19be0309a26848fc8643e6fa0150c7cc&scope.project.id=b8bbf8ceacf24390a9bcf23ca62622f7&role.id=fc68d072459d4720b9666c93bd535d63 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:13.263414 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee99470-1e9e-4d37-b5a1-a858b4571a4f None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:13.263885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee99470-1e9e-4d37-b5a1-a858b4571a4f None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:13.263977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee99470-1e9e-4d37-b5a1-a858b4571a4f None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:13.265329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 557/1107] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:13.270870 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c3e9c-011f-42c2-9194-507be9299abe None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:13.271277 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c3e9c-011f-42c2-9194-507be9299abe None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:13.271415 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c3e9c-011f-42c2-9194-507be9299abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:13.289777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-667c3e9c-011f-42c2-9194-507be9299abe None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:13.291009 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-667c3e9c-011f-42c2-9194-507be9299abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:13.354384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 551/1108] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:13.626244 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee99470-1e9e-4d37-b5a1-a858b4571a4f None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:13.723569 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee99470-1e9e-4d37-b5a1-a858b4571a4f None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:13.756261 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75983bc6-3247-40a6-8702-f4a0968b4045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:13.757382 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75983bc6-3247-40a6-8702-f4a0968b4045 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:13.757656 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75983bc6-3247-40a6-8702-f4a0968b4045 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:13.757924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75983bc6-3247-40a6-8702-f4a0968b4045 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:13.759048 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75983bc6-3247-40a6-8702-f4a0968b4045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:13.760477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75983bc6-3247-40a6-8702-f4a0968b4045 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:13.762009 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 558/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:37:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:13.767992 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c3e9c-011f-42c2-9194-507be9299abe None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:13.827546 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:13.872470 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c3e9c-011f-42c2-9194-507be9299abe None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:13.907055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24083cf5-9521-405e-837d-09a9de58bc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:13.907586 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24083cf5-9521-405e-837d-09a9de58bc7c None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:13.907864 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24083cf5-9521-405e-837d-09a9de58bc7c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:13.908135 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24083cf5-9521-405e-837d-09a9de58bc7c None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:13.908778 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24083cf5-9521-405e-837d-09a9de58bc7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:13.910233 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24083cf5-9521-405e-837d-09a9de58bc7c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:13.930523 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 552/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:37:13 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:37:16.146883 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34159430-9fc6-441f-a071-1dc495dbea91 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:16.147415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34159430-9fc6-441f-a071-1dc495dbea91 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:16.147527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34159430-9fc6-441f-a071-1dc495dbea91 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:16.148771 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 559/1111] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:16.153944 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd46ed-a717-4cbd-bfc8-334e6ace0b65 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:16.154368 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd46ed-a717-4cbd-bfc8-334e6ace0b65 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:16.154519 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd46ed-a717-4cbd-bfc8-334e6ace0b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:16.172861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-cfdd46ed-a717-4cbd-bfc8-334e6ace0b65 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:16.174063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cfdd46ed-a717-4cbd-bfc8-334e6ace0b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:16.238286 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 553/1112] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 12:37:16.508403 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34159430-9fc6-441f-a071-1dc495dbea91 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:16.545664 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:16.609583 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34159430-9fc6-441f-a071-1dc495dbea91 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:16.641817 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b771b0-e219-44ae-9dc6-8ca7a5eb0ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:16.642464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b771b0-e219-44ae-9dc6-8ca7a5eb0ea6 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:16.643002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b771b0-e219-44ae-9dc6-8ca7a5eb0ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:16.643272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b771b0-e219-44ae-9dc6-8ca7a5eb0ea6 None admin] PATH_INFO: `/v3/roles/ca13aabb28b6442f8540fa4e8bb1b35a` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:16.644654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b771b0-e219-44ae-9dc6-8ca7a5eb0ea6 None admin] RBAC: Authorizing `identity:get_role(role_id=ca13aabb28b6442f8540fa4e8bb1b35a)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:16.646011 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b771b0-e219-44ae-9dc6-8ca7a5eb0ea6 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:16.647867 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 560/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:16 2022] GET /identity/v3/roles/ca13aabb28b6442f8540fa4e8bb1b35a => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:16.654359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd46ed-a717-4cbd-bfc8-334e6ace0b65 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:16.702218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:16.751054 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd46ed-a717-4cbd-bfc8-334e6ace0b65 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:16.784388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b9692-badb-42ce-97f6-bfb13288a16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:16.784932 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b9692-badb-42ce-97f6-bfb13288a16f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:16.785202 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b9692-badb-42ce-97f6-bfb13288a16f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:16.785466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b9692-badb-42ce-97f6-bfb13288a16f None admin] PATH_INFO: `/v3/users/3ab01ff8b6244617814a20aa865e6aa2` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:16.798075 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b9692-badb-42ce-97f6-bfb13288a16f None admin] RBAC: Authorizing `identity:get_user(user_id=3ab01ff8b6244617814a20aa865e6aa2)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:16.799512 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b9692-badb-42ce-97f6-bfb13288a16f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:16.801106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 554/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:16 2022] GET /identity/v3/users/3ab01ff8b6244617814a20aa865e6aa2 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:16.807163 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b771b0-e219-44ae-9dc6-8ca7a5eb0ea6 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:16.906886 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b771b0-e219-44ae-9dc6-8ca7a5eb0ea6 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:16.909289 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:16.939947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b31cac-2494-482d-b840-694b31d75006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:16.940538 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b31cac-2494-482d-b840-694b31d75006 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:16.940813 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b31cac-2494-482d-b840-694b31d75006 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:16.941085 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b31cac-2494-482d-b840-694b31d75006 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:17.005788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b31cac-2494-482d-b840-694b31d75006 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:17.007817 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b31cac-2494-482d-b840-694b31d75006 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:17.103644 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 561/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:37:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:37:17.109545 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b9692-badb-42ce-97f6-bfb13288a16f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:17.208084 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b9692-badb-42ce-97f6-bfb13288a16f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:17.219466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:17.246222 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8e6b0-d023-4a1e-9c72-d4fb94c534d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:17.246852 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8e6b0-d023-4a1e-9c72-d4fb94c534d6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:17.247127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8e6b0-d023-4a1e-9c72-d4fb94c534d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:17.247396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8e6b0-d023-4a1e-9c72-d4fb94c534d6 None admin] PATH_INFO: `/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:17.248705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8e6b0-d023-4a1e-9c72-d4fb94c534d6 None admin] RBAC: Authorizing `identity:get_project(project_id=b8bbf8ceacf24390a9bcf23ca62622f7)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:17.250178 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8e6b0-d023-4a1e-9c72-d4fb94c534d6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:17.251899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 555/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:37:17 2022] GET /identity/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:17.258231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b31cac-2494-482d-b840-694b31d75006 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:17.261304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:17.356384 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b31cac-2494-482d-b840-694b31d75006 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:17.390258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3b24d-8511-489b-af2d-41aebe8a6817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:17.390852 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3b24d-8511-489b-af2d-41aebe8a6817 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:17.391126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3b24d-8511-489b-af2d-41aebe8a6817 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:17.391394 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3b24d-8511-489b-af2d-41aebe8a6817 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:17.392126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3b24d-8511-489b-af2d-41aebe8a6817 None admin] RBAC: Adding query filter params (role.id=ca13aabb28b6442f8540fa4e8bb1b35a, scope.project.id=b8bbf8ceacf24390a9bcf23ca62622f7, user.id=3ab01ff8b6244617814a20aa865e6aa2) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:17.392429 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3b24d-8511-489b-af2d-41aebe8a6817 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:17.394147 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3b24d-8511-489b-af2d-41aebe8a6817 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:17.400021 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 562/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:37:17 2022] GET /identity/v3/role_assignments?user.id=3ab01ff8b6244617814a20aa865e6aa2&scope.project.id=b8bbf8ceacf24390a9bcf23ca62622f7&role.id=ca13aabb28b6442f8540fa4e8bb1b35a => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:19.514288 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b566c0-e3b4-428a-b718-ff5a72a5bda9 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:19.514792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b566c0-e3b4-428a-b718-ff5a72a5bda9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:19.514792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b566c0-e3b4-428a-b718-ff5a72a5bda9 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:19.516055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 556/1118] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:19.521501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c369b-0d89-4854-97d6-042226b59a31 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:19.521912 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c369b-0d89-4854-97d6-042226b59a31 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:19.522065 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c369b-0d89-4854-97d6-042226b59a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:19.539782 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-b26c369b-0d89-4854-97d6-042226b59a31 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:19.540965 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-b26c369b-0d89-4854-97d6-042226b59a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:19.600210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:19.605860 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 563/1119] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:19.873204 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b566c0-e3b4-428a-b718-ff5a72a5bda9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:19.971854 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b566c0-e3b4-428a-b718-ff5a72a5bda9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:20.004166 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae613c31-3cc9-4794-9d4a-90be653faff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:20.004784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae613c31-3cc9-4794-9d4a-90be653faff8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:20.005055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae613c31-3cc9-4794-9d4a-90be653faff8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:20.005317 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae613c31-3cc9-4794-9d4a-90be653faff8 None admin] PATH_INFO: `/v3/roles/ca13aabb28b6442f8540fa4e8bb1b35a` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:20.007268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae613c31-3cc9-4794-9d4a-90be653faff8 None admin] RBAC: Authorizing `identity:get_role(role_id=ca13aabb28b6442f8540fa4e8bb1b35a)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:20.008615 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae613c31-3cc9-4794-9d4a-90be653faff8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:20.009917 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 557/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:19 2022] GET /identity/v3/roles/ca13aabb28b6442f8540fa4e8bb1b35a => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:20.015834 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c369b-0d89-4854-97d6-042226b59a31 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:20.111419 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c369b-0d89-4854-97d6-042226b59a31 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:20.145038 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0ec49-7d94-4e5a-9a6d-891fd59c0729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:20.145590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0ec49-7d94-4e5a-9a6d-891fd59c0729 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:20.145868 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0ec49-7d94-4e5a-9a6d-891fd59c0729 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:20.146167 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0ec49-7d94-4e5a-9a6d-891fd59c0729 None admin] PATH_INFO: `/v3/users/3ab01ff8b6244617814a20aa865e6aa2` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:20.147294 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0ec49-7d94-4e5a-9a6d-891fd59c0729 None admin] RBAC: Authorizing `identity:get_user(user_id=3ab01ff8b6244617814a20aa865e6aa2)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:20.148787 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0ec49-7d94-4e5a-9a6d-891fd59c0729 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:20.150387 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 564/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:20 2022] GET /identity/v3/users/3ab01ff8b6244617814a20aa865e6aa2 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:20.156131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae613c31-3cc9-4794-9d4a-90be653faff8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:20.200665 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:20.251661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae613c31-3cc9-4794-9d4a-90be653faff8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:20.285757 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d8b46-5279-4d13-b5ec-6f2be80a5889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:20.286404 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d8b46-5279-4d13-b5ec-6f2be80a5889 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:20.286675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d8b46-5279-4d13-b5ec-6f2be80a5889 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:20.286938 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d8b46-5279-4d13-b5ec-6f2be80a5889 None admin] PATH_INFO: `/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:20.288059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d8b46-5279-4d13-b5ec-6f2be80a5889 None admin] RBAC: Authorizing `identity:get_project(project_id=b8bbf8ceacf24390a9bcf23ca62622f7)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:20.289470 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d8b46-5279-4d13-b5ec-6f2be80a5889 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:20.291118 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 558/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:37:20 2022] GET /identity/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:20.297141 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0ec49-7d94-4e5a-9a6d-891fd59c0729 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:20.361378 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:20.397193 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0ec49-7d94-4e5a-9a6d-891fd59c0729 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:20.432679 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c876d-0644-46cc-9de6-84e6154f933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:20.433331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c876d-0644-46cc-9de6-84e6154f933a None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:20.433608 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c876d-0644-46cc-9de6-84e6154f933a None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:20.433877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c876d-0644-46cc-9de6-84e6154f933a None admin] PATH_INFO: `/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7/users/3ab01ff8b6244617814a20aa865e6aa2/roles/ca13aabb28b6442f8540fa4e8bb1b35a` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:20.436028 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c876d-0644-46cc-9de6-84e6154f933a None admin] RBAC: Authorizing `identity:create_grant(project_id=b8bbf8ceacf24390a9bcf23ca62622f7, user_id=3ab01ff8b6244617814a20aa865e6aa2, role_id=ca13aabb28b6442f8540fa4e8bb1b35a)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:20.437464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c876d-0644-46cc-9de6-84e6154f933a None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:20.450622 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 565/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:37:20 2022] PUT /identity/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7/users/3ab01ff8b6244617814a20aa865e6aa2/roles/ca13aabb28b6442f8540fa4e8bb1b35a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:37:22.594488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612b43d-7654-4904-b6a6-421c167043ea None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:22.595004 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612b43d-7654-4904-b6a6-421c167043ea None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:22.595004 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612b43d-7654-4904-b6a6-421c167043ea None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:22.596268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 559/1124] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:22.601440 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1730fbe-4fe0-4786-b727-49cf35f297cb None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:22.601808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1730fbe-4fe0-4786-b727-49cf35f297cb None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:22.601992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1730fbe-4fe0-4786-b727-49cf35f297cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:22.620333 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-b1730fbe-4fe0-4786-b727-49cf35f297cb None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:22.621527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-b1730fbe-4fe0-4786-b727-49cf35f297cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:22.687201 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 566/1125] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:22.954203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612b43d-7654-4904-b6a6-421c167043ea None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:23.051449 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612b43d-7654-4904-b6a6-421c167043ea None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:23.063952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:23.085305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e947f-5514-4e51-a259-d76be6052dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:23.085919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e947f-5514-4e51-a259-d76be6052dc3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:23.086354 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e947f-5514-4e51-a259-d76be6052dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:23.086626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e947f-5514-4e51-a259-d76be6052dc3 None admin] PATH_INFO: `/v3/roles/ca13aabb28b6442f8540fa4e8bb1b35a` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:23.088002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e947f-5514-4e51-a259-d76be6052dc3 None admin] RBAC: Authorizing `identity:get_role(role_id=ca13aabb28b6442f8540fa4e8bb1b35a)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:23.089354 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e947f-5514-4e51-a259-d76be6052dc3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:23.090687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 560/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:22 2022] GET /identity/v3/roles/ca13aabb28b6442f8540fa4e8bb1b35a => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:23.096478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1730fbe-4fe0-4786-b727-49cf35f297cb None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:23.150454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:23.193440 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1730fbe-4fe0-4786-b727-49cf35f297cb None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:23.228004 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4974e3-52a2-400d-aac8-c48682e6536c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:23.228531 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4974e3-52a2-400d-aac8-c48682e6536c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:23.228806 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4974e3-52a2-400d-aac8-c48682e6536c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:23.229075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4974e3-52a2-400d-aac8-c48682e6536c None admin] PATH_INFO: `/v3/users/3ab01ff8b6244617814a20aa865e6aa2` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:23.230225 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4974e3-52a2-400d-aac8-c48682e6536c None admin] RBAC: Authorizing `identity:get_user(user_id=3ab01ff8b6244617814a20aa865e6aa2)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:23.231662 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4974e3-52a2-400d-aac8-c48682e6536c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:23.233237 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 567/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:23 2022] GET /identity/v3/users/3ab01ff8b6244617814a20aa865e6aa2 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:23.239138 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e947f-5514-4e51-a259-d76be6052dc3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:23.337580 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e947f-5514-4e51-a259-d76be6052dc3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:23.373056 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa1c0e-ce5f-4f4f-81a9-dedc9f6406ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:23.373665 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa1c0e-ce5f-4f4f-81a9-dedc9f6406ef None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:23.373936 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa1c0e-ce5f-4f4f-81a9-dedc9f6406ef None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:23.374242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa1c0e-ce5f-4f4f-81a9-dedc9f6406ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:23.421144 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:23.440501 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa1c0e-ce5f-4f4f-81a9-dedc9f6406ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:23.441950 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa1c0e-ce5f-4f4f-81a9-dedc9f6406ef None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:23.535985 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 561/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:37:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:37:23.542623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4974e3-52a2-400d-aac8-c48682e6536c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:23.638011 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4974e3-52a2-400d-aac8-c48682e6536c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:23.660672 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:23.672210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a45a66-d649-49c7-af73-82e23299c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:23.672832 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a45a66-d649-49c7-af73-82e23299c211 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:23.673105 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a45a66-d649-49c7-af73-82e23299c211 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:23.673373 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a45a66-d649-49c7-af73-82e23299c211 None admin] PATH_INFO: `/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:23.674501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a45a66-d649-49c7-af73-82e23299c211 None admin] RBAC: Authorizing `identity:get_project(project_id=b8bbf8ceacf24390a9bcf23ca62622f7)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:23.675933 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a45a66-d649-49c7-af73-82e23299c211 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:23.677668 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 568/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:37:23 2022] GET /identity/v3/projects/b8bbf8ceacf24390a9bcf23ca62622f7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:23.684484 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa1c0e-ce5f-4f4f-81a9-dedc9f6406ef None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:23.762363 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:23.781677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa1c0e-ce5f-4f4f-81a9-dedc9f6406ef None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:23.815006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d175d-4bdc-41f1-9ac2-d33a8077ea63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:23.815528 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d175d-4bdc-41f1-9ac2-d33a8077ea63 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:23.815801 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d175d-4bdc-41f1-9ac2-d33a8077ea63 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:23.816065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d175d-4bdc-41f1-9ac2-d33a8077ea63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:23.816775 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d175d-4bdc-41f1-9ac2-d33a8077ea63 None admin] RBAC: Adding query filter params (role.id=ca13aabb28b6442f8540fa4e8bb1b35a, scope.project.id=b8bbf8ceacf24390a9bcf23ca62622f7, user.id=3ab01ff8b6244617814a20aa865e6aa2) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:23.817077 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d175d-4bdc-41f1-9ac2-d33a8077ea63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:23.818532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d175d-4bdc-41f1-9ac2-d33a8077ea63 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:23.824139 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 562/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:37:23 2022] GET /identity/v3/role_assignments?user.id=3ab01ff8b6244617814a20aa865e6aa2&scope.project.id=b8bbf8ceacf24390a9bcf23ca62622f7&role.id=ca13aabb28b6442f8540fa4e8bb1b35a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:26.014282 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1bed5-30bd-483f-9aee-49d48fa6664f None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:26.014855 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1bed5-30bd-483f-9aee-49d48fa6664f None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:26.014855 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1bed5-30bd-483f-9aee-49d48fa6664f None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:26.016151 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 569/1131] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:26.021630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487dbcf-acee-49cc-b824-32626f30aad2 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:26.022056 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487dbcf-acee-49cc-b824-32626f30aad2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:26.022258 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487dbcf-acee-49cc-b824-32626f30aad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:26.040651 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-2487dbcf-acee-49cc-b824-32626f30aad2 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:26.041850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2487dbcf-acee-49cc-b824-32626f30aad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:26.108125 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 563/1132] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:26.374734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1bed5-30bd-483f-9aee-49d48fa6664f None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:26.483988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1bed5-30bd-483f-9aee-49d48fa6664f None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:26.517044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325045d-d0b7-430e-898c-abe946f7eda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:26.517919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325045d-d0b7-430e-898c-abe946f7eda8 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:26.518225 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325045d-d0b7-430e-898c-abe946f7eda8 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:26.518499 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325045d-d0b7-430e-898c-abe946f7eda8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:26.519601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325045d-d0b7-430e-898c-abe946f7eda8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:26.521035 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325045d-d0b7-430e-898c-abe946f7eda8 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:26.523188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 570/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:37:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:26.529022 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487dbcf-acee-49cc-b824-32626f30aad2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:26.600651 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:26.625243 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487dbcf-acee-49cc-b824-32626f30aad2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:26.659481 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e27c6-53f3-4f78-b58a-3cd572ab6b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:26.660085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e27c6-53f3-4f78-b58a-3cd572ab6b17 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:26.660364 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e27c6-53f3-4f78-b58a-3cd572ab6b17 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:26.660636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e27c6-53f3-4f78-b58a-3cd572ab6b17 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:26.661278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e27c6-53f3-4f78-b58a-3cd572ab6b17 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:26.662736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e27c6-53f3-4f78-b58a-3cd572ab6b17 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:26.688930 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 564/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:37:26 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:37:28.869101 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3018c79-e752-4df9-8b2e-09ae7f804d43 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:28.869693 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3018c79-e752-4df9-8b2e-09ae7f804d43 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:28.869693 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3018c79-e752-4df9-8b2e-09ae7f804d43 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:28.870985 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 571/1135] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:28.876358 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc4a58-da6e-4609-944f-2db749ed8621 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:28.876805 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc4a58-da6e-4609-944f-2db749ed8621 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:28.876907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc4a58-da6e-4609-944f-2db749ed8621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:28.895593 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e9dc4a58-da6e-4609-944f-2db749ed8621 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:28.896818 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e9dc4a58-da6e-4609-944f-2db749ed8621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:28.961934 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 565/1136] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:29.227882 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3018c79-e752-4df9-8b2e-09ae7f804d43 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:29.289678 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:29.330571 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3018c79-e752-4df9-8b2e-09ae7f804d43 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:29.364358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adad877-8ac4-4e6c-a9c6-c71148f3fef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:29.364980 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adad877-8ac4-4e6c-a9c6-c71148f3fef3 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:29.365257 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adad877-8ac4-4e6c-a9c6-c71148f3fef3 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:29.365556 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adad877-8ac4-4e6c-a9c6-c71148f3fef3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:29.366697 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adad877-8ac4-4e6c-a9c6-c71148f3fef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:29.368119 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adad877-8ac4-4e6c-a9c6-c71148f3fef3 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:29.369673 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 572/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:37:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:29.375555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc4a58-da6e-4609-944f-2db749ed8621 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:29.430696 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:29.473058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc4a58-da6e-4609-944f-2db749ed8621 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:29.505689 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ca169-ce2d-44bf-af9a-87038ee52b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:29.507181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ca169-ce2d-44bf-af9a-87038ee52b1b None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:29.507462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ca169-ce2d-44bf-af9a-87038ee52b1b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:29.507729 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ca169-ce2d-44bf-af9a-87038ee52b1b None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:29.508382 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ca169-ce2d-44bf-af9a-87038ee52b1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:29.509810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ca169-ce2d-44bf-af9a-87038ee52b1b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:29.532171 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 566/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:37:29 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:37:31.723359 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03817e27-be43-44ca-aefe-3319734567ed None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:31.724147 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03817e27-be43-44ca-aefe-3319734567ed None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:31.724147 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03817e27-be43-44ca-aefe-3319734567ed None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:31.726008 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 573/1139] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:31.731341 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf3bbe-4607-40d7-b8d9-244634cd9518 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:31.731799 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf3bbe-4607-40d7-b8d9-244634cd9518 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:31.731903 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf3bbe-4607-40d7-b8d9-244634cd9518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:31.749466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-34cf3bbe-4607-40d7-b8d9-244634cd9518 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:31.750677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-34cf3bbe-4607-40d7-b8d9-244634cd9518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:31.815039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 567/1140] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:32.084747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03817e27-be43-44ca-aefe-3319734567ed None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:32.183580 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03817e27-be43-44ca-aefe-3319734567ed None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:32.209010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:32.217227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f902c7-439a-4ebd-bab2-18c7f94058a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:32.217848 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f902c7-439a-4ebd-bab2-18c7f94058a8 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:32.218156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f902c7-439a-4ebd-bab2-18c7f94058a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:32.218427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f902c7-439a-4ebd-bab2-18c7f94058a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:32.228824 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f902c7-439a-4ebd-bab2-18c7f94058a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-50f902c7-439a-4ebd-bab2-18c7f94058a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:37:32.233742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:32.235013 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:32.235970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 574/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:37:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:32.241920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf3bbe-4607-40d7-b8d9-244634cd9518 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:32.284593 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:32.340260 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf3bbe-4607-40d7-b8d9-244634cd9518 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:32.372544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a256e-def5-4524-b535-8f54e9c83cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:32.373137 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a256e-def5-4524-b535-8f54e9c83cb1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:32.373409 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a256e-def5-4524-b535-8f54e9c83cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:32.373676 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a256e-def5-4524-b535-8f54e9c83cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:32.374347 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a256e-def5-4524-b535-8f54e9c83cb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:32.374643 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a256e-def5-4524-b535-8f54e9c83cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:32.375988 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a256e-def5-4524-b535-8f54e9c83cb1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:32.384270 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 568/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:37:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:32.390274 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f902c7-439a-4ebd-bab2-18c7f94058a8 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:32.490017 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f902c7-439a-4ebd-bab2-18c7f94058a8 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:32.523852 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6484e-1512-497a-bfdb-f8a82b8d8cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:32.524472 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6484e-1512-497a-bfdb-f8a82b8d8cfa None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:32.524753 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6484e-1512-497a-bfdb-f8a82b8d8cfa None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:32.525025 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6484e-1512-497a-bfdb-f8a82b8d8cfa None admin] PATH_INFO: `/v3/users/724d1982e9bb47a8b5ea1c7280ae00db` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:32.539096 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6484e-1512-497a-bfdb-f8a82b8d8cfa None admin] RBAC: Authorizing `identity:get_user(user_id=724d1982e9bb47a8b5ea1c7280ae00db)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:32.540542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6484e-1512-497a-bfdb-f8a82b8d8cfa None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:32.542155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 575/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:32 2022] GET /identity/v3/users/724d1982e9bb47a8b5ea1c7280ae00db => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:32.548075 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a256e-def5-4524-b535-8f54e9c83cb1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:32.645002 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a256e-def5-4524-b535-8f54e9c83cb1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:32.647227 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:32.680398 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfba1d-0650-4149-9bd0-0676264770b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:32.681001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfba1d-0650-4149-9bd0-0676264770b7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:32.681277 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfba1d-0650-4149-9bd0-0676264770b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:32.681545 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfba1d-0650-4149-9bd0-0676264770b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:32.746970 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfba1d-0650-4149-9bd0-0676264770b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:32.748417 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dfba1d-0650-4149-9bd0-0676264770b7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:32.841707 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 569/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:37:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:37:32.847624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6484e-1512-497a-bfdb-f8a82b8d8cfa None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:32.893385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:32.945481 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6484e-1512-497a-bfdb-f8a82b8d8cfa None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:32.978526 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb47f65-1aa1-4376-a8c4-4a410c0c2384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:32.979152 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb47f65-1aa1-4376-a8c4-4a410c0c2384 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:32.979428 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb47f65-1aa1-4376-a8c4-4a410c0c2384 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:32.979697 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb47f65-1aa1-4376-a8c4-4a410c0c2384 None admin] PATH_INFO: `/v3/projects/dcc3e6b7bdc949c3baf512a7ce2e08f2` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:32.980815 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb47f65-1aa1-4376-a8c4-4a410c0c2384 None admin] RBAC: Authorizing `identity:get_project(project_id=dcc3e6b7bdc949c3baf512a7ce2e08f2)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:32.982270 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb47f65-1aa1-4376-a8c4-4a410c0c2384 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:32.983887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 576/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:37:32 2022] GET /identity/v3/projects/dcc3e6b7bdc949c3baf512a7ce2e08f2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:32.990180 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfba1d-0650-4149-9bd0-0676264770b7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:32.993232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:33.087024 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfba1d-0650-4149-9bd0-0676264770b7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:33.121025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd57aa-58b4-4674-b273-299fb20e9d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:33.121624 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd57aa-58b4-4674-b273-299fb20e9d89 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:33.121891 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd57aa-58b4-4674-b273-299fb20e9d89 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:33.122184 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd57aa-58b4-4674-b273-299fb20e9d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:33.122896 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd57aa-58b4-4674-b273-299fb20e9d89 None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=dcc3e6b7bdc949c3baf512a7ce2e08f2, user.id=724d1982e9bb47a8b5ea1c7280ae00db) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:33.123197 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd57aa-58b4-4674-b273-299fb20e9d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:33.124604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd57aa-58b4-4674-b273-299fb20e9d89 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:33.129825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 570/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:37:32 2022] GET /identity/v3/role_assignments?user.id=724d1982e9bb47a8b5ea1c7280ae00db&scope.project.id=dcc3e6b7bdc949c3baf512a7ce2e08f2&role.id=fc68d072459d4720b9666c93bd535d63 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:35.266336 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772330c-b629-4229-9f61-1a34f8af7233 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:35.266817 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772330c-b629-4229-9f61-1a34f8af7233 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:35.266917 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772330c-b629-4229-9f61-1a34f8af7233 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:35.268452 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 577/1147] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:35.274188 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b586d73-d08e-4f81-af51-6bbe5ff57fc5 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:35.274622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b586d73-d08e-4f81-af51-6bbe5ff57fc5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:35.274731 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b586d73-d08e-4f81-af51-6bbe5ff57fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:35.292935 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1b586d73-d08e-4f81-af51-6bbe5ff57fc5 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:35.294147 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1b586d73-d08e-4f81-af51-6bbe5ff57fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:35.353242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:35.359097 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 571/1148] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:35.631699 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772330c-b629-4229-9f61-1a34f8af7233 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:35.729205 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772330c-b629-4229-9f61-1a34f8af7233 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:35.742655 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:35.762877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd89eeb-c382-40dd-a9bf-a90a7fbcab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:35.763496 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd89eeb-c382-40dd-a9bf-a90a7fbcab93 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:35.763773 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd89eeb-c382-40dd-a9bf-a90a7fbcab93 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:35.764042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd89eeb-c382-40dd-a9bf-a90a7fbcab93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:35.774870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd89eeb-c382-40dd-a9bf-a90a7fbcab93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-ffd89eeb-c382-40dd-a9bf-a90a7fbcab93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:37:35.779547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:35.780617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:35.781456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 578/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:37:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:35.786939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b586d73-d08e-4f81-af51-6bbe5ff57fc5 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:35.884718 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b586d73-d08e-4f81-af51-6bbe5ff57fc5 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:35.917607 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de5dfd-e284-46b0-b33c-63b99b1978aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:35.918226 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de5dfd-e284-46b0-b33c-63b99b1978aa None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:35.918504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de5dfd-e284-46b0-b33c-63b99b1978aa None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:35.918770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de5dfd-e284-46b0-b33c-63b99b1978aa None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:35.919415 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de5dfd-e284-46b0-b33c-63b99b1978aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:35.919710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de5dfd-e284-46b0-b33c-63b99b1978aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:35.921059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de5dfd-e284-46b0-b33c-63b99b1978aa None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:35.929412 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 572/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:37:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:35.935259 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd89eeb-c382-40dd-a9bf-a90a7fbcab93 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:36.034395 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd89eeb-c382-40dd-a9bf-a90a7fbcab93 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:36.067138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63555200-aa30-437e-9cdb-7e9f872f4388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:36.067756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63555200-aa30-437e-9cdb-7e9f872f4388 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:36.068032 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63555200-aa30-437e-9cdb-7e9f872f4388 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:36.068349 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63555200-aa30-437e-9cdb-7e9f872f4388 None admin] PATH_INFO: `/v3/users/724d1982e9bb47a8b5ea1c7280ae00db` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:36.069957 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63555200-aa30-437e-9cdb-7e9f872f4388 None admin] RBAC: Authorizing `identity:get_user(user_id=724d1982e9bb47a8b5ea1c7280ae00db)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:36.071415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63555200-aa30-437e-9cdb-7e9f872f4388 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:36.072976 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 579/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:35 2022] GET /identity/v3/users/724d1982e9bb47a8b5ea1c7280ae00db => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:36.078845 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de5dfd-e284-46b0-b33c-63b99b1978aa None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:36.145846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:36.185787 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de5dfd-e284-46b0-b33c-63b99b1978aa None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:36.218855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3f037-dcef-4d05-b62a-bb2495c4b438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:36.219475 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3f037-dcef-4d05-b62a-bb2495c4b438 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:36.219752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3f037-dcef-4d05-b62a-bb2495c4b438 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:36.220025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3f037-dcef-4d05-b62a-bb2495c4b438 None admin] PATH_INFO: `/v3/projects/dcc3e6b7bdc949c3baf512a7ce2e08f2` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:36.221131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3f037-dcef-4d05-b62a-bb2495c4b438 None admin] RBAC: Authorizing `identity:get_project(project_id=dcc3e6b7bdc949c3baf512a7ce2e08f2)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:36.222597 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3f037-dcef-4d05-b62a-bb2495c4b438 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:36.224224 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 573/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:37:36 2022] GET /identity/v3/projects/dcc3e6b7bdc949c3baf512a7ce2e08f2 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:36.230791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63555200-aa30-437e-9cdb-7e9f872f4388 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:36.279663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:36.336182 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63555200-aa30-437e-9cdb-7e9f872f4388 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:36.369654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74d1da-d06b-4bfc-baec-3c6783dd625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:36.370336 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74d1da-d06b-4bfc-baec-3c6783dd625e None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:36.370618 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74d1da-d06b-4bfc-baec-3c6783dd625e None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:36.370893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74d1da-d06b-4bfc-baec-3c6783dd625e None admin] PATH_INFO: `/v3/projects/dcc3e6b7bdc949c3baf512a7ce2e08f2/users/724d1982e9bb47a8b5ea1c7280ae00db/roles/fc68d072459d4720b9666c93bd535d63` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:36.372808 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74d1da-d06b-4bfc-baec-3c6783dd625e None admin] RBAC: Authorizing `identity:create_grant(project_id=dcc3e6b7bdc949c3baf512a7ce2e08f2, user_id=724d1982e9bb47a8b5ea1c7280ae00db, role_id=fc68d072459d4720b9666c93bd535d63)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:36.374293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74d1da-d06b-4bfc-baec-3c6783dd625e None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:36.387366 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 580/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:37:36 2022] PUT /identity/v3/projects/dcc3e6b7bdc949c3baf512a7ce2e08f2/users/724d1982e9bb47a8b5ea1c7280ae00db/roles/fc68d072459d4720b9666c93bd535d63 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:37:38.524826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae78e4a-9b07-423d-87a4-b136f031e6eb None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:38.525330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae78e4a-9b07-423d-87a4-b136f031e6eb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:38.525330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae78e4a-9b07-423d-87a4-b136f031e6eb None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:38.526644 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 574/1154] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:38.531741 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca059e0-c00d-443a-a8bd-57de12901a4c None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:38.532178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca059e0-c00d-443a-a8bd-57de12901a4c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:38.532272 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca059e0-c00d-443a-a8bd-57de12901a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:38.550546 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-0ca059e0-c00d-443a-a8bd-57de12901a4c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:38.551734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-0ca059e0-c00d-443a-a8bd-57de12901a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:38.616241 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 581/1155] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:38.883463 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae78e4a-9b07-423d-87a4-b136f031e6eb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:38.984165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae78e4a-9b07-423d-87a4-b136f031e6eb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:39.010755 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:39.019184 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9d49e-9d2d-4142-beb4-b0a33146f085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:39.019808 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9d49e-9d2d-4142-beb4-b0a33146f085 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:39.020082 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9d49e-9d2d-4142-beb4-b0a33146f085 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:39.020350 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9d49e-9d2d-4142-beb4-b0a33146f085 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:39.030747 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9d49e-9d2d-4142-beb4-b0a33146f085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-77b9d49e-9d2d-4142-beb4-b0a33146f085 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:37:39.035418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:39.036568 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:37:39.037596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 575/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:37:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:39.042585 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca059e0-c00d-443a-a8bd-57de12901a4c None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:39.072452 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:39.144095 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca059e0-c00d-443a-a8bd-57de12901a4c None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:39.179930 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f782e21-6968-4467-8b7d-9f74662da99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:39.180533 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f782e21-6968-4467-8b7d-9f74662da99b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:39.180811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f782e21-6968-4467-8b7d-9f74662da99b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:39.181081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f782e21-6968-4467-8b7d-9f74662da99b None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:39.181733 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f782e21-6968-4467-8b7d-9f74662da99b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:39.182062 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f782e21-6968-4467-8b7d-9f74662da99b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:39.183437 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f782e21-6968-4467-8b7d-9f74662da99b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:39.191492 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 582/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:37:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:39.198587 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9d49e-9d2d-4142-beb4-b0a33146f085 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:39.296889 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9d49e-9d2d-4142-beb4-b0a33146f085 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:39.329952 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ecbf1-a952-4c3f-be41-17c41c4e5b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:39.330604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ecbf1-a952-4c3f-be41-17c41c4e5b58 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:39.330871 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ecbf1-a952-4c3f-be41-17c41c4e5b58 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:39.331212 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ecbf1-a952-4c3f-be41-17c41c4e5b58 None admin] PATH_INFO: `/v3/users/724d1982e9bb47a8b5ea1c7280ae00db` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:39.332234 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ecbf1-a952-4c3f-be41-17c41c4e5b58 None admin] RBAC: Authorizing `identity:get_user(user_id=724d1982e9bb47a8b5ea1c7280ae00db)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:39.333650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ecbf1-a952-4c3f-be41-17c41c4e5b58 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:39.335832 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 576/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:39 2022] GET /identity/v3/users/724d1982e9bb47a8b5ea1c7280ae00db => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:39.341645 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f782e21-6968-4467-8b7d-9f74662da99b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:39.432446 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:39.439258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f782e21-6968-4467-8b7d-9f74662da99b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:39.472266 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f625d0-eb9b-4cce-896d-c8a9fe1483bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:39.472862 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f625d0-eb9b-4cce-896d-c8a9fe1483bf None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:39.473137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f625d0-eb9b-4cce-896d-c8a9fe1483bf None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:39.473408 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f625d0-eb9b-4cce-896d-c8a9fe1483bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:39.538980 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f625d0-eb9b-4cce-896d-c8a9fe1483bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:39.540425 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f625d0-eb9b-4cce-896d-c8a9fe1483bf None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:39.627333 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:39.635121 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 583/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:37:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:37:39.641055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ecbf1-a952-4c3f-be41-17c41c4e5b58 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:39.698139 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:39.741230 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ecbf1-a952-4c3f-be41-17c41c4e5b58 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:39.775854 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65385f0-13fc-4d03-b2ad-4663e8e8da2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:39.776488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65385f0-13fc-4d03-b2ad-4663e8e8da2d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:39.776766 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65385f0-13fc-4d03-b2ad-4663e8e8da2d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:39.777038 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65385f0-13fc-4d03-b2ad-4663e8e8da2d None admin] PATH_INFO: `/v3/projects/dcc3e6b7bdc949c3baf512a7ce2e08f2` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:39.778195 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65385f0-13fc-4d03-b2ad-4663e8e8da2d None admin] RBAC: Authorizing `identity:get_project(project_id=dcc3e6b7bdc949c3baf512a7ce2e08f2)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:39.779626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65385f0-13fc-4d03-b2ad-4663e8e8da2d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:39.781265 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 577/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:37:39 2022] GET /identity/v3/projects/dcc3e6b7bdc949c3baf512a7ce2e08f2 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:39.787629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f625d0-eb9b-4cce-896d-c8a9fe1483bf None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:39.884580 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f625d0-eb9b-4cce-896d-c8a9fe1483bf None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:39.918205 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d08ae-7e82-4632-8453-e95863e89869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:39.918803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d08ae-7e82-4632-8453-e95863e89869 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:39.919077 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d08ae-7e82-4632-8453-e95863e89869 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:39.919343 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d08ae-7e82-4632-8453-e95863e89869 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:39.920055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d08ae-7e82-4632-8453-e95863e89869 None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=dcc3e6b7bdc949c3baf512a7ce2e08f2, user.id=724d1982e9bb47a8b5ea1c7280ae00db) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:39.920357 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d08ae-7e82-4632-8453-e95863e89869 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:39.921785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d08ae-7e82-4632-8453-e95863e89869 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:39.928038 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 584/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:37:39 2022] GET /identity/v3/role_assignments?user.id=724d1982e9bb47a8b5ea1c7280ae00db&scope.project.id=dcc3e6b7bdc949c3baf512a7ce2e08f2&role.id=fc68d072459d4720b9666c93bd535d63 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:42.108164 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e52e8-27de-40ad-83e7-6df96459ac1c None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:42.108725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e52e8-27de-40ad-83e7-6df96459ac1c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:42.108725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e52e8-27de-40ad-83e7-6df96459ac1c None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:42.110026 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 578/1162] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:42.115445 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13231f6c-b468-4caf-96e5-7679d06baa60 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:42.115858 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13231f6c-b468-4caf-96e5-7679d06baa60 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:42.115975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13231f6c-b468-4caf-96e5-7679d06baa60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:42.134540 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-13231f6c-b468-4caf-96e5-7679d06baa60 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:42.135710 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-13231f6c-b468-4caf-96e5-7679d06baa60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:42.192685 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:42.207056 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 585/1163] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:42.472500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e52e8-27de-40ad-83e7-6df96459ac1c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:42.571898 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e52e8-27de-40ad-83e7-6df96459ac1c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:42.594196 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:42.604898 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860070ee-e6d7-4e4f-ba30-fd53e772373c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:42.605499 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860070ee-e6d7-4e4f-ba30-fd53e772373c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:42.605775 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860070ee-e6d7-4e4f-ba30-fd53e772373c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:42.606059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860070ee-e6d7-4e4f-ba30-fd53e772373c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:42.672158 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860070ee-e6d7-4e4f-ba30-fd53e772373c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:42.673636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860070ee-e6d7-4e4f-ba30-fd53e772373c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:42.773527 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 579/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:37:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:37:42.779457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13231f6c-b468-4caf-96e5-7679d06baa60 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:42.880715 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13231f6c-b468-4caf-96e5-7679d06baa60 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:42.914545 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937676c-c530-4e5a-ad4e-8e428008f10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:42.915163 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937676c-c530-4e5a-ad4e-8e428008f10f None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:42.915437 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937676c-c530-4e5a-ad4e-8e428008f10f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:42.915705 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937676c-c530-4e5a-ad4e-8e428008f10f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:42.922178 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937676c-c530-4e5a-ad4e-8e428008f10f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:42.923617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937676c-c530-4e5a-ad4e-8e428008f10f None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-7937676c-c530-4e5a-ad4e-8e428008f10f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:42.935227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:42.936353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:37:42.937663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 586/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Tue Oct 4 12:37:42 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:42.942407 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860070ee-e6d7-4e4f-ba30-fd53e772373c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:42.966883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:43.039287 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860070ee-e6d7-4e4f-ba30-fd53e772373c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:43.071452 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e28e95-01ff-437b-9f08-4c4f1f2f6a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:43.072070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e28e95-01ff-437b-9f08-4c4f1f2f6a66 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:43.072344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e28e95-01ff-437b-9f08-4c4f1f2f6a66 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:43.072609 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e28e95-01ff-437b-9f08-4c4f1f2f6a66 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:43.073255 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e28e95-01ff-437b-9f08-4c4f1f2f6a66 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:43.073549 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e28e95-01ff-437b-9f08-4c4f1f2f6a66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:43.074926 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e28e95-01ff-437b-9f08-4c4f1f2f6a66 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:43.081720 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 580/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Tue Oct 4 12:37:42 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:45.208233 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51d99c-c8e3-43da-b7ea-a50667a01f40 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:45.209309 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51d99c-c8e3-43da-b7ea-a50667a01f40 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:45.209309 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51d99c-c8e3-43da-b7ea-a50667a01f40 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:45.210691 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 587/1167] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:45.216004 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a6c8f-0bf6-4700-a6d0-6f8c2221eb9c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:45.216399 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a6c8f-0bf6-4700-a6d0-6f8c2221eb9c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:45.216546 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a6c8f-0bf6-4700-a6d0-6f8c2221eb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:45.234192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ff7a6c8f-0bf6-4700-a6d0-6f8c2221eb9c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:45.235361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ff7a6c8f-0bf6-4700-a6d0-6f8c2221eb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:45.304939 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 581/1168] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:45.572373 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51d99c-c8e3-43da-b7ea-a50667a01f40 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:45.620816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:45.671468 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51d99c-c8e3-43da-b7ea-a50667a01f40 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:45.704734 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0c63e-13c8-4758-9e40-4dad44949fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:45.705329 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0c63e-13c8-4758-9e40-4dad44949fb3 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:45.705601 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0c63e-13c8-4758-9e40-4dad44949fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:45.705866 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0c63e-13c8-4758-9e40-4dad44949fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:45.706561 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0c63e-13c8-4758-9e40-4dad44949fb3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:45.707910 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0c63e-13c8-4758-9e40-4dad44949fb3 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:45.724089 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 588/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Tue Oct 4 12:37:45 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:37:47.898308 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abca5d4-b52b-41d0-8429-0ddfb9e45ca1 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:47.899367 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abca5d4-b52b-41d0-8429-0ddfb9e45ca1 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:47.899367 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abca5d4-b52b-41d0-8429-0ddfb9e45ca1 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:47.900161 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 582/1170] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:47.906081 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddcb92-25db-453a-9733-2f5b88db0ae0 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:47.906526 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddcb92-25db-453a-9733-2f5b88db0ae0 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:47.906615 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddcb92-25db-453a-9733-2f5b88db0ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:47.925319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-bfddcb92-25db-453a-9733-2f5b88db0ae0 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:47.926561 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-bfddcb92-25db-453a-9733-2f5b88db0ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:47.996978 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 589/1171] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:48.262055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abca5d4-b52b-41d0-8429-0ddfb9e45ca1 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:48.281102 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:48.363625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abca5d4-b52b-41d0-8429-0ddfb9e45ca1 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:48.397777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54908f65-1b12-4f43-8c7d-1ac1c1540c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:48.398501 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54908f65-1b12-4f43-8c7d-1ac1c1540c60 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:48.398776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54908f65-1b12-4f43-8c7d-1ac1c1540c60 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:48.399046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54908f65-1b12-4f43-8c7d-1ac1c1540c60 None admin] PATH_INFO: `/v3/domains/161b2c83ca1a49eca933a140017d3239` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:48.409593 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54908f65-1b12-4f43-8c7d-1ac1c1540c60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=161b2c83ca1a49eca933a140017d3239)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:48.411083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54908f65-1b12-4f43-8c7d-1ac1c1540c60 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:48.413376 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 583/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Tue Oct 4 12:37:48 2022] GET /identity/v3/domains/161b2c83ca1a49eca933a140017d3239 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:48.419572 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddcb92-25db-453a-9733-2f5b88db0ae0 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:48.457769 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:48.517930 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddcb92-25db-453a-9733-2f5b88db0ae0 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:48.551769 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6adc7d-e321-4cb2-a425-6238d1fe0228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:48.552397 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6adc7d-e321-4cb2-a425-6238d1fe0228 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:48.552672 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6adc7d-e321-4cb2-a425-6238d1fe0228 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:48.552989 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6adc7d-e321-4cb2-a425-6238d1fe0228 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:48.553597 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6adc7d-e321-4cb2-a425-6238d1fe0228 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:48.555060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6adc7d-e321-4cb2-a425-6238d1fe0228 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:48.583056 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 590/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:37:48 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:37:50.806191 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c01f34-f477-4043-9643-68ff247811fa None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:50.806758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c01f34-f477-4043-9643-68ff247811fa None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:50.806758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c01f34-f477-4043-9643-68ff247811fa None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:50.808070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 584/1174] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:50.813574 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daedd4-6073-4af7-8a54-252793ddadbd None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:50.814080 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daedd4-6073-4af7-8a54-252793ddadbd None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:50.814194 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92daedd4-6073-4af7-8a54-252793ddadbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:50.833402 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-92daedd4-6073-4af7-8a54-252793ddadbd None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:50.834646 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-92daedd4-6073-4af7-8a54-252793ddadbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:50.905694 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 591/1175] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:51.177610 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c01f34-f477-4043-9643-68ff247811fa None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:51.276034 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:51.305172 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c01f34-f477-4043-9643-68ff247811fa None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:51.344104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687300e2-60b5-4f8f-becf-2e53c576c757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:51.344760 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687300e2-60b5-4f8f-becf-2e53c576c757 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:51.345039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687300e2-60b5-4f8f-becf-2e53c576c757 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:51.345311 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687300e2-60b5-4f8f-becf-2e53c576c757 None admin] PATH_INFO: `/v3/domains/161b2c83ca1a49eca933a140017d3239` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:51.346578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687300e2-60b5-4f8f-becf-2e53c576c757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=161b2c83ca1a49eca933a140017d3239)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:51.348063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687300e2-60b5-4f8f-becf-2e53c576c757 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:51.349789 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 585/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Tue Oct 4 12:37:51 2022] GET /identity/v3/domains/161b2c83ca1a49eca933a140017d3239 => generated 267 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:51.356062 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daedd4-6073-4af7-8a54-252793ddadbd None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:51.388460 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:51.499418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92daedd4-6073-4af7-8a54-252793ddadbd None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:51.543032 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e19ac-c18c-49ce-ac2c-09fc349128e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:51.543637 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e19ac-c18c-49ce-ac2c-09fc349128e7 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:51.543910 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e19ac-c18c-49ce-ac2c-09fc349128e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:51.544177 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e19ac-c18c-49ce-ac2c-09fc349128e7 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:51.544820 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e19ac-c18c-49ce-ac2c-09fc349128e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:51.546286 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e19ac-c18c-49ce-ac2c-09fc349128e7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:51.571026 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 592/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:37:51 2022] POST /identity/v3/users => generated 311 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 12:37:53.809887 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1614b7-8163-40fc-b7c7-a4c42b0fc870 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:53.810495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1614b7-8163-40fc-b7c7-a4c42b0fc870 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:53.810495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1614b7-8163-40fc-b7c7-a4c42b0fc870 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:53.811929 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 586/1178] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:53.817182 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075cb22-ab94-45c3-9a9a-8e7d419e1d03 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:53.817590 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075cb22-ab94-45c3-9a9a-8e7d419e1d03 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:53.817745 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075cb22-ab94-45c3-9a9a-8e7d419e1d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:53.836026 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-4075cb22-ab94-45c3-9a9a-8e7d419e1d03 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:53.837202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-4075cb22-ab94-45c3-9a9a-8e7d419e1d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:53.910209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 593/1179] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:54.181079 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1614b7-8163-40fc-b7c7-a4c42b0fc870 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:54.282360 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1614b7-8163-40fc-b7c7-a4c42b0fc870 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:54.315398 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d7f1-1902-4e6d-a79c-de78d5647f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:54.316013 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d7f1-1902-4e6d-a79c-de78d5647f74 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:54.316286 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d7f1-1902-4e6d-a79c-de78d5647f74 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:54.316554 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73d7f1-1902-4e6d-a79c-de78d5647f74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:54.326990 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73d7f1-1902-4e6d-a79c-de78d5647f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-dd73d7f1-1902-4e6d-a79c-de78d5647f74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:37:54.331656 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:54.332748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:37:54.333636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 587/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:37:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:54.338870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075cb22-ab94-45c3-9a9a-8e7d419e1d03 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:54.346986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:54.438229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075cb22-ab94-45c3-9a9a-8e7d419e1d03 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:54.471885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223cb02-e135-4b53-a523-3640e38897a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:54.472492 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223cb02-e135-4b53-a523-3640e38897a5 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:54.472770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223cb02-e135-4b53-a523-3640e38897a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:54.473042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223cb02-e135-4b53-a523-3640e38897a5 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:54.473701 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223cb02-e135-4b53-a523-3640e38897a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:54.474002 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223cb02-e135-4b53-a523-3640e38897a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:54.475410 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223cb02-e135-4b53-a523-3640e38897a5 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:54.483695 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 594/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:37:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:54.490289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d7f1-1902-4e6d-a79c-de78d5647f74 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:54.499578 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:54.599171 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73d7f1-1902-4e6d-a79c-de78d5647f74 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:54.638124 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e592ff0-ca78-4271-8710-b990b2c6c275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:54.638764 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e592ff0-ca78-4271-8710-b990b2c6c275 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:54.639037 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e592ff0-ca78-4271-8710-b990b2c6c275 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:54.639307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e592ff0-ca78-4271-8710-b990b2c6c275 None admin] PATH_INFO: `/v3/users/9df4376ef04341e9b68f1162a7c1179c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:54.654837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e592ff0-ca78-4271-8710-b990b2c6c275 None admin] RBAC: Authorizing `identity:get_user(user_id=9df4376ef04341e9b68f1162a7c1179c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:54.656319 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e592ff0-ca78-4271-8710-b990b2c6c275 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:54.658088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 588/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:54 2022] GET /identity/v3/users/9df4376ef04341e9b68f1162a7c1179c => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:54.664583 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223cb02-e135-4b53-a523-3640e38897a5 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:54.736738 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:54.773010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223cb02-e135-4b53-a523-3640e38897a5 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:54.808568 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2087153-dee6-44f4-a2b7-3ad680bab121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:54.809192 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2087153-dee6-44f4-a2b7-3ad680bab121 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:54.809466 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2087153-dee6-44f4-a2b7-3ad680bab121 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:54.809735 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2087153-dee6-44f4-a2b7-3ad680bab121 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:54.880208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2087153-dee6-44f4-a2b7-3ad680bab121 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:54.881709 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2087153-dee6-44f4-a2b7-3ad680bab121 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:54.947787 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:54.983606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 595/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:37:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:37:54.991547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e592ff0-ca78-4271-8710-b990b2c6c275 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:55.068264 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:55.108000 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e592ff0-ca78-4271-8710-b990b2c6c275 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:55.141172 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e90470-7bd2-4832-94bb-f01beb784ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:55.142080 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e90470-7bd2-4832-94bb-f01beb784ddf None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:55.142354 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e90470-7bd2-4832-94bb-f01beb784ddf None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:55.142618 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e90470-7bd2-4832-94bb-f01beb784ddf None admin] PATH_INFO: `/v3/projects/487733f578934bf482a5bdcf7ec36503` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:55.143722 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e90470-7bd2-4832-94bb-f01beb784ddf None admin] RBAC: Authorizing `identity:get_project(project_id=487733f578934bf482a5bdcf7ec36503)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:55.145145 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e90470-7bd2-4832-94bb-f01beb784ddf None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:55.147285 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 589/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:37:54 2022] GET /identity/v3/projects/487733f578934bf482a5bdcf7ec36503 => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:55.153411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2087153-dee6-44f4-a2b7-3ad680bab121 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:55.251312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2087153-dee6-44f4-a2b7-3ad680bab121 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:55.287434 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe9443-e41f-4517-b36e-14896f7f9731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:55.288035 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe9443-e41f-4517-b36e-14896f7f9731 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:55.288410 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe9443-e41f-4517-b36e-14896f7f9731 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:55.288718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe9443-e41f-4517-b36e-14896f7f9731 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:55.289520 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe9443-e41f-4517-b36e-14896f7f9731 None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=487733f578934bf482a5bdcf7ec36503, user.id=9df4376ef04341e9b68f1162a7c1179c) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:55.289936 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe9443-e41f-4517-b36e-14896f7f9731 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:55.291739 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe9443-e41f-4517-b36e-14896f7f9731 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:55.302791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 596/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:37:55 2022] GET /identity/v3/role_assignments?user.id=9df4376ef04341e9b68f1162a7c1179c&scope.project.id=487733f578934bf482a5bdcf7ec36503&role.id=fc68d072459d4720b9666c93bd535d63 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:57.477710 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8640c79-2d43-43ae-9b90-89f2e116db31 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:57.478289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8640c79-2d43-43ae-9b90-89f2e116db31 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:57.478289 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8640c79-2d43-43ae-9b90-89f2e116db31 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:57.479647 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 590/1186] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:37:57.485621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985399-4cf5-4f66-bff4-168977c9e212 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:57.486188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985399-4cf5-4f66-bff4-168977c9e212 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:57.486188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985399-4cf5-4f66-bff4-168977c9e212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:57.504924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-7b985399-4cf5-4f66-bff4-168977c9e212 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:37:57.506176 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-7b985399-4cf5-4f66-bff4-168977c9e212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:37:57.535271 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:57.579229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 597/1187] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:37:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:37:57.847017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8640c79-2d43-43ae-9b90-89f2e116db31 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:57.954526 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8640c79-2d43-43ae-9b90-89f2e116db31 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:57.985785 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:57.991312 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56377d-57ec-4648-bd3e-83a82fc774f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:57.991964 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56377d-57ec-4648-bd3e-83a82fc774f0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:57.992244 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56377d-57ec-4648-bd3e-83a82fc774f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:57.992518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56377d-57ec-4648-bd3e-83a82fc774f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:58.003962 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56377d-57ec-4648-bd3e-83a82fc774f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-8a56377d-57ec-4648-bd3e-83a82fc774f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:37:58.009396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:37:58.010574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:37:58.015154 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 591/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:37:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:37:58.017271 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b985399-4cf5-4f66-bff4-168977c9e212 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:58.119344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b985399-4cf5-4f66-bff4-168977c9e212 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:58.154462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18801669-4b98-49b5-b929-851adb02b94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:58.155069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18801669-4b98-49b5-b929-851adb02b94b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:58.155342 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18801669-4b98-49b5-b929-851adb02b94b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:58.155609 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18801669-4b98-49b5-b929-851adb02b94b None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:58.156263 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18801669-4b98-49b5-b929-851adb02b94b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:37:58.156559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18801669-4b98-49b5-b929-851adb02b94b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:58.157922 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18801669-4b98-49b5-b929-851adb02b94b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:58.166140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 598/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:37:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:58.172086 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56377d-57ec-4648-bd3e-83a82fc774f0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:58.271078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56377d-57ec-4648-bd3e-83a82fc774f0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:58.304254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb44ef7-7ede-4ed2-94de-d66c0892cfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:58.304869 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb44ef7-7ede-4ed2-94de-d66c0892cfd1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:58.305140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb44ef7-7ede-4ed2-94de-d66c0892cfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:58.305404 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb44ef7-7ede-4ed2-94de-d66c0892cfd1 None admin] PATH_INFO: `/v3/users/9df4376ef04341e9b68f1162a7c1179c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:58.306540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb44ef7-7ede-4ed2-94de-d66c0892cfd1 None admin] RBAC: Authorizing `identity:get_user(user_id=9df4376ef04341e9b68f1162a7c1179c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:58.307951 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb44ef7-7ede-4ed2-94de-d66c0892cfd1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:58.309506 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 592/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:37:58 2022] GET /identity/v3/users/9df4376ef04341e9b68f1162a7c1179c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:58.315445 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18801669-4b98-49b5-b929-851adb02b94b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:58.340686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:58.414798 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18801669-4b98-49b5-b929-851adb02b94b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:58.448312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bf48c-5b85-4061-939b-c81fee2d8b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:58.448935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bf48c-5b85-4061-939b-c81fee2d8b40 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:58.449217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bf48c-5b85-4061-939b-c81fee2d8b40 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:58.449489 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bf48c-5b85-4061-939b-c81fee2d8b40 None admin] PATH_INFO: `/v3/projects/487733f578934bf482a5bdcf7ec36503` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:58.450625 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bf48c-5b85-4061-939b-c81fee2d8b40 None admin] RBAC: Authorizing `identity:get_project(project_id=487733f578934bf482a5bdcf7ec36503)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:58.452073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bf48c-5b85-4061-939b-c81fee2d8b40 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:58.453706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 599/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:37:58 2022] GET /identity/v3/projects/487733f578934bf482a5bdcf7ec36503 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:37:58.459738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb44ef7-7ede-4ed2-94de-d66c0892cfd1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:37:58.517695 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:37:58.558055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb44ef7-7ede-4ed2-94de-d66c0892cfd1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:37:58.590588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2c34d-d621-4d24-9449-d0901cc1db3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:37:58.591239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2c34d-d621-4d24-9449-d0901cc1db3f None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:37:58.591515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2c34d-d621-4d24-9449-d0901cc1db3f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:37:58.591785 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2c34d-d621-4d24-9449-d0901cc1db3f None admin] PATH_INFO: `/v3/projects/487733f578934bf482a5bdcf7ec36503/users/9df4376ef04341e9b68f1162a7c1179c/roles/fc68d072459d4720b9666c93bd535d63` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:37:58.593670 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2c34d-d621-4d24-9449-d0901cc1db3f None admin] RBAC: Authorizing `identity:create_grant(project_id=487733f578934bf482a5bdcf7ec36503, user_id=9df4376ef04341e9b68f1162a7c1179c, role_id=fc68d072459d4720b9666c93bd535d63)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:37:58.595131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2c34d-d621-4d24-9449-d0901cc1db3f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:37:58.607831 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 593/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:37:58 2022] PUT /identity/v3/projects/487733f578934bf482a5bdcf7ec36503/users/9df4376ef04341e9b68f1162a7c1179c/roles/fc68d072459d4720b9666c93bd535d63 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:38:00.759767 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d095d-80d5-4a07-8b43-f2fc706cfa7c None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:38:00.760345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d095d-80d5-4a07-8b43-f2fc706cfa7c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:38:00.760345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d095d-80d5-4a07-8b43-f2fc706cfa7c None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:38:00.761637 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 600/1193] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:38:00.767296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201f528-580a-4220-8b89-e861ed22849c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:38:00.767729 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201f528-580a-4220-8b89-e861ed22849c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:38:00.767826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201f528-580a-4220-8b89-e861ed22849c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:38:00.786331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9201f528-580a-4220-8b89-e861ed22849c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:38:00.787523 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9201f528-580a-4220-8b89-e861ed22849c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:38:00.851671 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 594/1194] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:38:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:38:01.120910 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d095d-80d5-4a07-8b43-f2fc706cfa7c None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:38:01.218057 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:38:01.228613 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d095d-80d5-4a07-8b43-f2fc706cfa7c None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:38:01.268432 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3251b51-cc85-459f-8668-668c1ba3974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:38:01.269075 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3251b51-cc85-459f-8668-668c1ba3974f None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:38:01.269353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3251b51-cc85-459f-8668-668c1ba3974f None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:38:01.269625 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3251b51-cc85-459f-8668-668c1ba3974f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:38:01.280387 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3251b51-cc85-459f-8668-668c1ba3974f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-d3251b51-cc85-459f-8668-668c1ba3974f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:38:01.285129 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:38:01.286354 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:38:01.287269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 601/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:38:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:38:01.292388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201f528-580a-4220-8b89-e861ed22849c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:38:01.340743 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:38:01.392147 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201f528-580a-4220-8b89-e861ed22849c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:38:01.427259 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c7cb7-eb95-41ca-96dc-f83da9672000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:38:01.427863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c7cb7-eb95-41ca-96dc-f83da9672000 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:38:01.428138 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c7cb7-eb95-41ca-96dc-f83da9672000 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:38:01.428404 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c7cb7-eb95-41ca-96dc-f83da9672000 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:38:01.429057 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c7cb7-eb95-41ca-96dc-f83da9672000 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:38:01.429359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c7cb7-eb95-41ca-96dc-f83da9672000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:38:01.430739 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c7cb7-eb95-41ca-96dc-f83da9672000 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:38:01.438839 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 595/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:38:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:38:01.444887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3251b51-cc85-459f-8668-668c1ba3974f None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:38:01.540237 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3251b51-cc85-459f-8668-668c1ba3974f None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:38:01.573775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226298b-6327-4727-a9d8-157d0d1774a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:38:01.574410 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226298b-6327-4727-a9d8-157d0d1774a6 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:38:01.574684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226298b-6327-4727-a9d8-157d0d1774a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:38:01.574947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226298b-6327-4727-a9d8-157d0d1774a6 None admin] PATH_INFO: `/v3/users/9df4376ef04341e9b68f1162a7c1179c` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:38:01.576060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226298b-6327-4727-a9d8-157d0d1774a6 None admin] RBAC: Authorizing `identity:get_user(user_id=9df4376ef04341e9b68f1162a7c1179c)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:38:01.577469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226298b-6327-4727-a9d8-157d0d1774a6 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:38:01.579054 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 602/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Tue Oct 4 12:38:01 2022] GET /identity/v3/users/9df4376ef04341e9b68f1162a7c1179c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:38:01.585145 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c7cb7-eb95-41ca-96dc-f83da9672000 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:38:01.684138 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c7cb7-eb95-41ca-96dc-f83da9672000 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:38:01.696483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:38:01.717215 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734fa73-f816-44ae-a32f-c7458d6566e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:38:01.717821 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734fa73-f816-44ae-a32f-c7458d6566e4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:38:01.718120 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734fa73-f816-44ae-a32f-c7458d6566e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:38:01.718390 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734fa73-f816-44ae-a32f-c7458d6566e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:38:01.784275 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734fa73-f816-44ae-a32f-c7458d6566e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:38:01.785715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734fa73-f816-44ae-a32f-c7458d6566e4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:38:01.881391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 596/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:38:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 12:38:01.887429 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226298b-6327-4727-a9d8-157d0d1774a6 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:38:01.909751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:38:01.997223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226298b-6327-4727-a9d8-157d0d1774a6 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:38:02.034357 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82cee3-2bb7-458b-a8e9-701f0d6d1f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:38:02.034981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82cee3-2bb7-458b-a8e9-701f0d6d1f68 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:38:02.035256 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82cee3-2bb7-458b-a8e9-701f0d6d1f68 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:38:02.035524 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82cee3-2bb7-458b-a8e9-701f0d6d1f68 None admin] PATH_INFO: `/v3/projects/487733f578934bf482a5bdcf7ec36503` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:38:02.037191 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82cee3-2bb7-458b-a8e9-701f0d6d1f68 None admin] RBAC: Authorizing `identity:get_project(project_id=487733f578934bf482a5bdcf7ec36503)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:38:02.038657 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82cee3-2bb7-458b-a8e9-701f0d6d1f68 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:38:02.040279 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 603/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Tue Oct 4 12:38:01 2022] GET /identity/v3/projects/487733f578934bf482a5bdcf7ec36503 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:38:02.046415 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734fa73-f816-44ae-a32f-c7458d6566e4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:38:02.060260 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:38:02.142891 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734fa73-f816-44ae-a32f-c7458d6566e4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:38:02.177001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b919e-0c59-4af7-8ae5-2c6a638421ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:38:02.177604 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b919e-0c59-4af7-8ae5-2c6a638421ae None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:38:02.177879 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b919e-0c59-4af7-8ae5-2c6a638421ae None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:38:02.178179 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b919e-0c59-4af7-8ae5-2c6a638421ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:38:02.178903 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b919e-0c59-4af7-8ae5-2c6a638421ae None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=487733f578934bf482a5bdcf7ec36503, user.id=9df4376ef04341e9b68f1162a7c1179c) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:38:02.179206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b919e-0c59-4af7-8ae5-2c6a638421ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:38:02.180629 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b919e-0c59-4af7-8ae5-2c6a638421ae None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:38:02.186436 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 597/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:38:02 2022] GET /identity/v3/role_assignments?user.id=9df4376ef04341e9b68f1162a7c1179c&scope.project.id=487733f578934bf482a5bdcf7ec36503&role.id=fc68d072459d4720b9666c93bd535d63 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:38:12.030139 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=56 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:38:12.183768 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=62 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:00.113372 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7bd0-2cc6-41ba-aade-f615d8632d07 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:00.113976 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7bd0-2cc6-41ba-aade-f615d8632d07 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:00.113976 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7bd0-2cc6-41ba-aade-f615d8632d07 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:00.115700 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 604/1201] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:39:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:39:00.122851 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaae4a-2578-46e7-b3eb-c9abd41b96ae None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:00.123475 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaae4a-2578-46e7-b3eb-c9abd41b96ae None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:00.123794 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaae4a-2578-46e7-b3eb-c9abd41b96ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:00.151475 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6aaaae4a-2578-46e7-b3eb-c9abd41b96ae None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:39:00.153810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6aaaae4a-2578-46e7-b3eb-c9abd41b96ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:39:00.318094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 598/1202] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:39:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:39:00.637492 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b7bd0-2cc6-41ba-aade-f615d8632d07 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:00.785054 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b7bd0-2cc6-41ba-aade-f615d8632d07 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:00.834369 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef98bbc-4cb4-48b6-9db2-e1ead98293b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:00.835021 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef98bbc-4cb4-48b6-9db2-e1ead98293b9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:00.835424 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef98bbc-4cb4-48b6-9db2-e1ead98293b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:00.835640 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef98bbc-4cb4-48b6-9db2-e1ead98293b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:00.859785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef98bbc-4cb4-48b6-9db2-e1ead98293b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:00.861605 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef98bbc-4cb4-48b6-9db2-e1ead98293b9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-6ef98bbc-4cb4-48b6-9db2-e1ead98293b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:00.888594 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:00.894188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:00.895504 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:00.895504 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:00.895504 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:00.895504 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 605/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:39:00.897819 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaae4a-2578-46e7-b3eb-c9abd41b96ae None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:01.060598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaae4a-2578-46e7-b3eb-c9abd41b96ae None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:01.121058 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:01.130236 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec77ae-7a74-4b70-877b-280af7e8af05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:01.131305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec77ae-7a74-4b70-877b-280af7e8af05 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:01.131305 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec77ae-7a74-4b70-877b-280af7e8af05 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:01.131873 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec77ae-7a74-4b70-877b-280af7e8af05 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:01.132307 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec77ae-7a74-4b70-877b-280af7e8af05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:01.132713 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec77ae-7a74-4b70-877b-280af7e8af05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:01.134388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec77ae-7a74-4b70-877b-280af7e8af05 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:01.150677 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 599/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:01.156976 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef98bbc-4cb4-48b6-9db2-e1ead98293b9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:01.247469 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:01.322988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef98bbc-4cb4-48b6-9db2-e1ead98293b9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:01.372975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a77ec3-88f6-4f72-a4b3-8d776b1e6483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:01.373817 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a77ec3-88f6-4f72-a4b3-8d776b1e6483 None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:01.374142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a77ec3-88f6-4f72-a4b3-8d776b1e6483 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:01.374417 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a77ec3-88f6-4f72-a4b3-8d776b1e6483 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:01.375215 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a77ec3-88f6-4f72-a4b3-8d776b1e6483 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:01.377096 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a77ec3-88f6-4f72-a4b3-8d776b1e6483 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:01.422579 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 606/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Tue Oct 4 12:39:01 2022] POST /identity/v3/users => generated 251 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:39:03.866491 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d197461-2e17-426a-8810-6dec96a305eb None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:03.867078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d197461-2e17-426a-8810-6dec96a305eb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:03.867078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d197461-2e17-426a-8810-6dec96a305eb None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:03.868457 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 600/1206] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:39:03.875086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb8243-cdce-4699-b1bc-4ed1fbc03d36 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:03.875677 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb8243-cdce-4699-b1bc-4ed1fbc03d36 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:03.875677 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb8243-cdce-4699-b1bc-4ed1fbc03d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:03.897064 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-47bb8243-cdce-4699-b1bc-4ed1fbc03d36 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:39:03.898479 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-47bb8243-cdce-4699-b1bc-4ed1fbc03d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:39:03.997316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 607/1207] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:39:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:39:04.317315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d197461-2e17-426a-8810-6dec96a305eb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:04.447249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d197461-2e17-426a-8810-6dec96a305eb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:04.493032 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8678f-56b4-4ade-87d5-7b23c7d00b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:04.493649 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8678f-56b4-4ade-87d5-7b23c7d00b31 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:04.494134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8678f-56b4-4ade-87d5-7b23c7d00b31 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:04.494244 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8678f-56b4-4ade-87d5-7b23c7d00b31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:04.509979 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8678f-56b4-4ade-87d5-7b23c7d00b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-67b8678f-56b4-4ade-87d5-7b23c7d00b31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:39:04.523175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:04.524453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:04.525613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:04.525613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:04.525613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:04.525613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:04.525613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:39:04.525613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:39:04.525613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:39:04.525613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:39:04.525613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:39:04.525613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:39:04.525613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:39:04.525613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:04.531268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 601/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:39:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:04.541344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb8243-cdce-4699-b1bc-4ed1fbc03d36 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:04.615554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:04.699970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb8243-cdce-4699-b1bc-4ed1fbc03d36 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:04.753051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c314432-6c76-4b4b-a1da-f34d4f4c557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:04.753776 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c314432-6c76-4b4b-a1da-f34d4f4c557d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:04.754085 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c314432-6c76-4b4b-a1da-f34d4f4c557d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:04.754441 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c314432-6c76-4b4b-a1da-f34d4f4c557d None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:04.755160 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c314432-6c76-4b4b-a1da-f34d4f4c557d None admin] RBAC: Adding query filter params (name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:04.755485 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c314432-6c76-4b4b-a1da-f34d4f4c557d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:04.756874 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c314432-6c76-4b4b-a1da-f34d4f4c557d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:04.766303 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 608/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:39:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:04.773825 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8678f-56b4-4ade-87d5-7b23c7d00b31 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:04.859191 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:04.923666 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8678f-56b4-4ade-87d5-7b23c7d00b31 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:04.968526 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21928967-2483-4cdb-80d7-a66e852baa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:04.969180 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21928967-2483-4cdb-80d7-a66e852baa29 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:04.969454 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21928967-2483-4cdb-80d7-a66e852baa29 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:04.971085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21928967-2483-4cdb-80d7-a66e852baa29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:04.983017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21928967-2483-4cdb-80d7-a66e852baa29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:04.984515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21928967-2483-4cdb-80d7-a66e852baa29 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-21928967-2483-4cdb-80d7-a66e852baa29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:05.001401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:05.003296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:05.004321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:05.004321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:05.004321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:05.004321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 602/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:05.009529 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c314432-6c76-4b4b-a1da-f34d4f4c557d None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:05.147713 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:05.151077 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c314432-6c76-4b4b-a1da-f34d4f4c557d None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:05.201678 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63912b98-c6b4-46fc-8f3d-5eb6edd4b233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:05.202319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63912b98-c6b4-46fc-8f3d-5eb6edd4b233 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:05.202597 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63912b98-c6b4-46fc-8f3d-5eb6edd4b233 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:05.202898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63912b98-c6b4-46fc-8f3d-5eb6edd4b233 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:05.203506 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63912b98-c6b4-46fc-8f3d-5eb6edd4b233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:05.203877 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63912b98-c6b4-46fc-8f3d-5eb6edd4b233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:05.205854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63912b98-c6b4-46fc-8f3d-5eb6edd4b233 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:05.219447 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 609/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:39:05.227290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21928967-2483-4cdb-80d7-a66e852baa29 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:05.388911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21928967-2483-4cdb-80d7-a66e852baa29 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:05.392751 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:05.460544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43580ac-c6fc-4022-b9f5-b9ffa906b3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:05.461522 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43580ac-c6fc-4022-b9f5-b9ffa906b3b2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:05.462251 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43580ac-c6fc-4022-b9f5-b9ffa906b3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:05.462708 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43580ac-c6fc-4022-b9f5-b9ffa906b3b2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:05.478460 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43580ac-c6fc-4022-b9f5-b9ffa906b3b2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:05.479968 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43580ac-c6fc-4022-b9f5-b9ffa906b3b2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-f43580ac-c6fc-4022-b9f5-b9ffa906b3b2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:39:05.493837 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:05.495715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:05.497488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:05.497488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:05.497488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:39:05.497488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:39:05.497488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:39:05.497488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:39:05.497488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:39:05.497488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:39:05.497488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 12:39:05.497488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:05.497488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 603/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:05.501474 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63912b98-c6b4-46fc-8f3d-5eb6edd4b233 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:05.649455 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63912b98-c6b4-46fc-8f3d-5eb6edd4b233 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:05.694893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e2666-5883-418c-aa80-44cef0181ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:05.695723 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e2666-5883-418c-aa80-44cef0181ce2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:05.696095 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e2666-5883-418c-aa80-44cef0181ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:05.696541 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e2666-5883-418c-aa80-44cef0181ce2 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:05.698087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e2666-5883-418c-aa80-44cef0181ce2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:05.698087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e2666-5883-418c-aa80-44cef0181ce2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:05.699557 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e2666-5883-418c-aa80-44cef0181ce2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:05.729896 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 610/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Tue Oct 4 12:39:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:05.736846 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43580ac-c6fc-4022-b9f5-b9ffa906b3b2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:05.900093 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43580ac-c6fc-4022-b9f5-b9ffa906b3b2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:05.939000 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d4a33-ecad-4dd2-93b9-784bb04f1626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:05.941820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d4a33-ecad-4dd2-93b9-784bb04f1626 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:05.941820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d4a33-ecad-4dd2-93b9-784bb04f1626 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:05.941820 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d4a33-ecad-4dd2-93b9-784bb04f1626 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:05.990382 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:06.044684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d4a33-ecad-4dd2-93b9-784bb04f1626 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:06.046421 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d4a33-ecad-4dd2-93b9-784bb04f1626 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:06.180538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 604/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:39:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:39:06.186173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e2666-5883-418c-aa80-44cef0181ce2 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:06.211583 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:06.375463 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e2666-5883-418c-aa80-44cef0181ce2 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:06.421875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0af30-7d98-4fa4-9e6b-67e0791622b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:06.422575 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0af30-7d98-4fa4-9e6b-67e0791622b6 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:06.422855 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0af30-7d98-4fa4-9e6b-67e0791622b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:06.423132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0af30-7d98-4fa4-9e6b-67e0791622b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:06.436014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0af30-7d98-4fa4-9e6b-67e0791622b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:06.437812 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0af30-7d98-4fa4-9e6b-67e0791622b6 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-1cc0af30-7d98-4fa4-9e6b-67e0791622b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:06.456042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:06.458183 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:06.460312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:06.460312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:06.460312 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:06.462258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 611/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:06.464720 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d4a33-ecad-4dd2-93b9-784bb04f1626 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:06.560454 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:06.612836 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d4a33-ecad-4dd2-93b9-784bb04f1626 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:06.675468 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec743a8f-b159-4c31-a4c6-a06c420dd558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:06.676379 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec743a8f-b159-4c31-a4c6-a06c420dd558 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:06.676693 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec743a8f-b159-4c31-a4c6-a06c420dd558 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:06.677125 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec743a8f-b159-4c31-a4c6-a06c420dd558 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:06.678059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec743a8f-b159-4c31-a4c6-a06c420dd558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:06.678481 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec743a8f-b159-4c31-a4c6-a06c420dd558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:06.680626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec743a8f-b159-4c31-a4c6-a06c420dd558 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:06.695444 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 605/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:06.710953 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0af30-7d98-4fa4-9e6b-67e0791622b6 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:06.812068 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:06.854733 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0af30-7d98-4fa4-9e6b-67e0791622b6 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:06.899516 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97d217-30d3-44d8-91e3-b095624c01fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:06.900435 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97d217-30d3-44d8-91e3-b095624c01fa None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:06.900817 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97d217-30d3-44d8-91e3-b095624c01fa None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:06.901119 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97d217-30d3-44d8-91e3-b095624c01fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:06.912245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97d217-30d3-44d8-91e3-b095624c01fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:06.914141 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97d217-30d3-44d8-91e3-b095624c01fa None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-bb97d217-30d3-44d8-91e3-b095624c01fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:39:06.934559 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:06.936816 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:06.938844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:39:06.938844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:39:06.938844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:39:06.938844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:39:06.938844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:39:06.938844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:39:06.938844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:39:06.938844 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:06.941711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 612/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:39:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:06.947591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec743a8f-b159-4c31-a4c6-a06c420dd558 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:07.076142 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec743a8f-b159-4c31-a4c6-a06c420dd558 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:07.103062 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:07.116761 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7c921-cdb2-4819-a6a7-c75ea41a9fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:07.117367 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7c921-cdb2-4819-a6a7-c75ea41a9fdd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:07.117600 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7c921-cdb2-4819-a6a7-c75ea41a9fdd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:07.117882 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7c921-cdb2-4819-a6a7-c75ea41a9fdd None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:07.118615 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7c921-cdb2-4819-a6a7-c75ea41a9fdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:07.118913 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7c921-cdb2-4819-a6a7-c75ea41a9fdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:07.120453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7c921-cdb2-4819-a6a7-c75ea41a9fdd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:07.134447 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 606/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:39:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:07.141346 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97d217-30d3-44d8-91e3-b095624c01fa None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:07.296772 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97d217-30d3-44d8-91e3-b095624c01fa None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:07.339742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:07.348111 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da003fbd-0ddd-4cad-807c-1a3373fd7b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:07.348647 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da003fbd-0ddd-4cad-807c-1a3373fd7b88 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:07.348854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da003fbd-0ddd-4cad-807c-1a3373fd7b88 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:07.349181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da003fbd-0ddd-4cad-807c-1a3373fd7b88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:07.349910 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da003fbd-0ddd-4cad-807c-1a3373fd7b88 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=3a442a785ebe45709bce78add0e0e93a) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:07.350242 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da003fbd-0ddd-4cad-807c-1a3373fd7b88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:07.351836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da003fbd-0ddd-4cad-807c-1a3373fd7b88 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:07.358332 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 613/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:39:07 2022] GET /identity/v3/role_assignments?user.id=3a442a785ebe45709bce78add0e0e93a&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:09.725273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b022bad-8d77-4bbd-8e62-d37160166e1b None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:09.725768 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b022bad-8d77-4bbd-8e62-d37160166e1b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:09.725768 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b022bad-8d77-4bbd-8e62-d37160166e1b None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:09.727388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 607/1220] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:39:09.734212 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254152b-ae08-41c9-8436-478df3d5ab26 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:09.734646 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254152b-ae08-41c9-8436-478df3d5ab26 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:09.734753 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254152b-ae08-41c9-8436-478df3d5ab26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:09.754410 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-c254152b-ae08-41c9-8436-478df3d5ab26 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:39:09.755641 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-c254152b-ae08-41c9-8436-478df3d5ab26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:39:09.830896 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 614/1221] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:39:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:39:10.102900 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b022bad-8d77-4bbd-8e62-d37160166e1b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:10.211177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b022bad-8d77-4bbd-8e62-d37160166e1b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:10.248702 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6531e00-592e-4ba5-b23b-80b8b8038c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:10.249342 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6531e00-592e-4ba5-b23b-80b8b8038c59 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:10.252337 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6531e00-592e-4ba5-b23b-80b8b8038c59 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:10.252337 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6531e00-592e-4ba5-b23b-80b8b8038c59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:10.263041 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6531e00-592e-4ba5-b23b-80b8b8038c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-e6531e00-592e-4ba5-b23b-80b8b8038c59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:39:10.270916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:10.271947 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:10.272783 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 608/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:39:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:10.279123 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254152b-ae08-41c9-8436-478df3d5ab26 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:10.387334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254152b-ae08-41c9-8436-478df3d5ab26 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:10.418227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:10.434173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d0e87-028d-41e7-aacb-95981bffa87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:10.435160 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d0e87-028d-41e7-aacb-95981bffa87c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:10.435536 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d0e87-028d-41e7-aacb-95981bffa87c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:10.436071 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d0e87-028d-41e7-aacb-95981bffa87c None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:10.436814 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d0e87-028d-41e7-aacb-95981bffa87c None admin] RBAC: Adding query filter params (name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:10.437298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d0e87-028d-41e7-aacb-95981bffa87c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:10.439138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d0e87-028d-41e7-aacb-95981bffa87c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:10.456460 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 615/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:39:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:10.457504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6531e00-592e-4ba5-b23b-80b8b8038c59 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:10.515141 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:10.586784 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6531e00-592e-4ba5-b23b-80b8b8038c59 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:10.646475 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba8338-b931-465a-9df3-66f6d52ecccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:10.647376 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba8338-b931-465a-9df3-66f6d52ecccd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:10.647704 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba8338-b931-465a-9df3-66f6d52ecccd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:10.647978 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba8338-b931-465a-9df3-66f6d52ecccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:10.661085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba8338-b931-465a-9df3-66f6d52ecccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:10.662733 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba8338-b931-465a-9df3-66f6d52ecccd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-32ba8338-b931-465a-9df3-66f6d52ecccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:10.679081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:10.680414 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:10.681393 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:10.681393 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:10.681393 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:10.681393 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 609/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:10.686227 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d0e87-028d-41e7-aacb-95981bffa87c None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:10.796945 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d0e87-028d-41e7-aacb-95981bffa87c None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:10.838016 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd578866-1079-4c15-817c-c26890447a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:10.838665 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd578866-1079-4c15-817c-c26890447a23 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:10.838917 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd578866-1079-4c15-817c-c26890447a23 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:10.839331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd578866-1079-4c15-817c-c26890447a23 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:10.839851 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd578866-1079-4c15-817c-c26890447a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:10.840142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd578866-1079-4c15-817c-c26890447a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:10.841502 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd578866-1079-4c15-817c-c26890447a23 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:10.854465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 616/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:10.860392 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba8338-b931-465a-9df3-66f6d52ecccd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:10.966087 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:10.977403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba8338-b931-465a-9df3-66f6d52ecccd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:11.020209 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f824cff-e702-4841-b691-a6428987fc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:11.020919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f824cff-e702-4841-b691-a6428987fc5d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:11.021136 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f824cff-e702-4841-b691-a6428987fc5d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:11.021373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f824cff-e702-4841-b691-a6428987fc5d None admin] PATH_INFO: `/v3/users/placement` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:11.031141 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f824cff-e702-4841-b691-a6428987fc5d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:11.034080 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f824cff-e702-4841-b691-a6428987fc5d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-1f824cff-e702-4841-b691-a6428987fc5d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:39:11.050630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:11.052931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:11.055544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:11.055544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:11.055544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:39:11.055544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:39:11.055544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:39:11.055544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:39:11.055544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:39:11.055544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:39:11.055544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 12:39:11.055544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:11.055544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 610/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:11.061571 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd578866-1079-4c15-817c-c26890447a23 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:11.118418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:11.184917 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd578866-1079-4c15-817c-c26890447a23 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:11.221190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdac64-1431-4cd0-97c8-74790c5bd62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:11.221682 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdac64-1431-4cd0-97c8-74790c5bd62d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:11.221961 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdac64-1431-4cd0-97c8-74790c5bd62d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:11.222276 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdac64-1431-4cd0-97c8-74790c5bd62d None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:11.223206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfdac64-1431-4cd0-97c8-74790c5bd62d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:11.223496 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfdac64-1431-4cd0-97c8-74790c5bd62d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:11.225186 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfdac64-1431-4cd0-97c8-74790c5bd62d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:11.248237 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 617/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Tue Oct 4 12:39:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:11.255353 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f824cff-e702-4841-b691-a6428987fc5d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:11.378842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f824cff-e702-4841-b691-a6428987fc5d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:11.419520 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fe2eb-40aa-4574-8f36-ef29e6fbc5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:11.420227 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fe2eb-40aa-4574-8f36-ef29e6fbc5ed None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:11.420498 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fe2eb-40aa-4574-8f36-ef29e6fbc5ed None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:11.420764 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fe2eb-40aa-4574-8f36-ef29e6fbc5ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:11.434786 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fe2eb-40aa-4574-8f36-ef29e6fbc5ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:11.436332 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fe2eb-40aa-4574-8f36-ef29e6fbc5ed None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-5b1fe2eb-40aa-4574-8f36-ef29e6fbc5ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:11.454762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:11.456736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:11.458486 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:11.458486 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:11.458486 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:11.458486 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 611/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:11.463128 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdac64-1431-4cd0-97c8-74790c5bd62d None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:11.592291 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdac64-1431-4cd0-97c8-74790c5bd62d None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:11.594776 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:11.632121 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036adda-ea4c-4430-910e-c475e8b61acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:11.632882 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036adda-ea4c-4430-910e-c475e8b61acf None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:11.633162 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036adda-ea4c-4430-910e-c475e8b61acf None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:11.633456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036adda-ea4c-4430-910e-c475e8b61acf None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:11.634184 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036adda-ea4c-4430-910e-c475e8b61acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:11.634487 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036adda-ea4c-4430-910e-c475e8b61acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:11.636154 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036adda-ea4c-4430-910e-c475e8b61acf None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:11.651015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 618/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:11.656810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fe2eb-40aa-4574-8f36-ef29e6fbc5ed None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:11.667941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:11.772591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fe2eb-40aa-4574-8f36-ef29e6fbc5ed None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:11.809736 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b6cc9-3d60-4052-a65f-044b0526a0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:11.810401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b6cc9-3d60-4052-a65f-044b0526a0c7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:11.810678 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b6cc9-3d60-4052-a65f-044b0526a0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:11.810949 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b6cc9-3d60-4052-a65f-044b0526a0c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:11.818030 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b6cc9-3d60-4052-a65f-044b0526a0c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:11.819502 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b6cc9-3d60-4052-a65f-044b0526a0c7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-624b6cc9-3d60-4052-a65f-044b0526a0c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:39:11.832033 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:11.833192 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:11.834225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:39:11.834225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:39:11.834225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:39:11.834225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:39:11.834225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:39:11.834225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:39:11.834225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:39:11.834225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:11.834225 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 612/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:39:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:39:11.840206 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036adda-ea4c-4430-910e-c475e8b61acf None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:11.972829 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036adda-ea4c-4430-910e-c475e8b61acf None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:12.024504 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515df82d-a220-449d-bc2b-69e148e31c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:12.025168 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df82d-a220-449d-bc2b-69e148e31c58 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:12.025446 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df82d-a220-449d-bc2b-69e148e31c58 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:12.025715 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df82d-a220-449d-bc2b-69e148e31c58 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:12.026516 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515df82d-a220-449d-bc2b-69e148e31c58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:12.026942 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515df82d-a220-449d-bc2b-69e148e31c58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:12.028755 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515df82d-a220-449d-bc2b-69e148e31c58 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:12.046026 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 619/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:39:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:12.053591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b6cc9-3d60-4052-a65f-044b0526a0c7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:12.129639 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:12.184156 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b6cc9-3d60-4052-a65f-044b0526a0c7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:12.221612 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c790ec-dc68-4a41-8081-74ff45fe205e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:12.222308 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c790ec-dc68-4a41-8081-74ff45fe205e None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:12.222590 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c790ec-dc68-4a41-8081-74ff45fe205e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:12.222909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c790ec-dc68-4a41-8081-74ff45fe205e None admin] PATH_INFO: `/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/3a442a785ebe45709bce78add0e0e93a/roles/943fc0edf23046a69ba6102f2c1e544c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:12.240684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c790ec-dc68-4a41-8081-74ff45fe205e None admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fb003a695490faa0077bbbe5e7d77, user_id=3a442a785ebe45709bce78add0e0e93a, role_id=943fc0edf23046a69ba6102f2c1e544c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:12.242393 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c790ec-dc68-4a41-8081-74ff45fe205e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:12.257463 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 613/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:39:12 2022] PUT /identity/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/3a442a785ebe45709bce78add0e0e93a/roles/943fc0edf23046a69ba6102f2c1e544c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:39:14.569554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7499cc0-7441-489b-90c5-939bae56ce9f None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:14.570124 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7499cc0-7441-489b-90c5-939bae56ce9f None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:14.570124 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7499cc0-7441-489b-90c5-939bae56ce9f None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:14.571475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 620/1233] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:39:14.577296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e978879-3c8f-4a48-85e3-65b35bd4f447 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:14.577776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e978879-3c8f-4a48-85e3-65b35bd4f447 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:14.577874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e978879-3c8f-4a48-85e3-65b35bd4f447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:14.598702 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-2e978879-3c8f-4a48-85e3-65b35bd4f447 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:39:14.599919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2e978879-3c8f-4a48-85e3-65b35bd4f447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:39:14.677543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 614/1234] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:39:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:39:14.967648 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7499cc0-7441-489b-90c5-939bae56ce9f None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:15.019023 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:15.120766 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7499cc0-7441-489b-90c5-939bae56ce9f None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:15.169964 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc2de4-c9a8-47dd-af26-f2ff49ef179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:15.171088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc2de4-c9a8-47dd-af26-f2ff49ef179d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:15.171567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc2de4-c9a8-47dd-af26-f2ff49ef179d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:15.171987 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc2de4-c9a8-47dd-af26-f2ff49ef179d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:15.189749 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc2de4-c9a8-47dd-af26-f2ff49ef179d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-e3bc2de4-c9a8-47dd-af26-f2ff49ef179d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:39:15.196777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:15.198030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:15.198898 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 621/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Tue Oct 4 12:39:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:15.205276 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e978879-3c8f-4a48-85e3-65b35bd4f447 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:15.255111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:15.312821 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e978879-3c8f-4a48-85e3-65b35bd4f447 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:15.349531 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfafa8c-afc8-44ef-a852-b4e195c2252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:15.350185 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfafa8c-afc8-44ef-a852-b4e195c2252f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:15.350499 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfafa8c-afc8-44ef-a852-b4e195c2252f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:15.350769 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfafa8c-afc8-44ef-a852-b4e195c2252f None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:15.351427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfafa8c-afc8-44ef-a852-b4e195c2252f None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:15.351722 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfafa8c-afc8-44ef-a852-b4e195c2252f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:15.353093 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfafa8c-afc8-44ef-a852-b4e195c2252f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:15.361717 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 615/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Tue Oct 4 12:39:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:15.368541 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc2de4-c9a8-47dd-af26-f2ff49ef179d None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:15.467695 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:15.486413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc2de4-c9a8-47dd-af26-f2ff49ef179d None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:15.524316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df399bc-955d-465b-a2e0-af7011a01bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:15.525061 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df399bc-955d-465b-a2e0-af7011a01bcc None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:15.525306 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df399bc-955d-465b-a2e0-af7011a01bcc None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:15.525654 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df399bc-955d-465b-a2e0-af7011a01bcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:15.538134 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df399bc-955d-465b-a2e0-af7011a01bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:15.539939 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df399bc-955d-465b-a2e0-af7011a01bcc None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-1df399bc-955d-465b-a2e0-af7011a01bcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:15.557231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:15.558542 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:15.560052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:15.560052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:15.560052 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:15.561420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 622/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:15.568532 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfafa8c-afc8-44ef-a852-b4e195c2252f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:15.696369 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfafa8c-afc8-44ef-a852-b4e195c2252f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:15.708574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:15.738420 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafef6d-2421-448a-bb46-12d207809c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:15.739411 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafef6d-2421-448a-bb46-12d207809c53 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:15.739694 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafef6d-2421-448a-bb46-12d207809c53 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:15.740231 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafef6d-2421-448a-bb46-12d207809c53 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:15.740887 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fafef6d-2421-448a-bb46-12d207809c53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:15.741187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fafef6d-2421-448a-bb46-12d207809c53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:15.742865 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fafef6d-2421-448a-bb46-12d207809c53 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:15.759453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 616/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:15.766426 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df399bc-955d-465b-a2e0-af7011a01bcc None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:15.881603 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df399bc-955d-465b-a2e0-af7011a01bcc None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:15.922944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0580e-0dd4-4382-96b9-f5bb8d1c1129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:15.923611 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0580e-0dd4-4382-96b9-f5bb8d1c1129 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:15.923967 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0580e-0dd4-4382-96b9-f5bb8d1c1129 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:15.924242 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0580e-0dd4-4382-96b9-f5bb8d1c1129 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:15.928159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:15.934323 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0580e-0dd4-4382-96b9-f5bb8d1c1129 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:15.935903 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0580e-0dd4-4382-96b9-f5bb8d1c1129 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-15e0580e-0dd4-4382-96b9-f5bb8d1c1129 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:39:15.950087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:15.951224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:15.952187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:15.952187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:15.952187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:39:15.952187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:39:15.952187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:39:15.952187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:39:15.952187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:39:15.952187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:39:15.952187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 12:39:15.952187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:15.952187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 623/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:15.957626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafef6d-2421-448a-bb46-12d207809c53 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:16.075250 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafef6d-2421-448a-bb46-12d207809c53 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:16.110768 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e8fcf-a63e-4d45-b3cf-e64d806c2f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:16.111355 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8fcf-a63e-4d45-b3cf-e64d806c2f74 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:16.111631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8fcf-a63e-4d45-b3cf-e64d806c2f74 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:16.111900 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8fcf-a63e-4d45-b3cf-e64d806c2f74 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:16.112630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e8fcf-a63e-4d45-b3cf-e64d806c2f74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:16.112927 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e8fcf-a63e-4d45-b3cf-e64d806c2f74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:16.114391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e8fcf-a63e-4d45-b3cf-e64d806c2f74 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:16.134342 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 617/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Tue Oct 4 12:39:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:16.140258 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0580e-0dd4-4382-96b9-f5bb8d1c1129 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:16.276189 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0580e-0dd4-4382-96b9-f5bb8d1c1129 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:16.317851 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15e52-2a2a-42ff-b878-ba6528cb0ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:16.318533 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15e52-2a2a-42ff-b878-ba6528cb0ca7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:16.318778 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15e52-2a2a-42ff-b878-ba6528cb0ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:16.319157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15e52-2a2a-42ff-b878-ba6528cb0ca7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:16.388652 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:16.398213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb15e52-2a2a-42ff-b878-ba6528cb0ca7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:16.400038 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb15e52-2a2a-42ff-b878-ba6528cb0ca7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:16.517131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 624/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:39:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:39:16.523858 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e8fcf-a63e-4d45-b3cf-e64d806c2f74 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:16.555377 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:16.650366 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e8fcf-a63e-4d45-b3cf-e64d806c2f74 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:16.689836 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5dccd-6621-4b45-9645-09133755a28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:16.690814 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5dccd-6621-4b45-9645-09133755a28b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:16.691347 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5dccd-6621-4b45-9645-09133755a28b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:16.691638 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5dccd-6621-4b45-9645-09133755a28b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:16.706187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5dccd-6621-4b45-9645-09133755a28b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:16.707934 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5dccd-6621-4b45-9645-09133755a28b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-1bf5dccd-6621-4b45-9645-09133755a28b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:16.726256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:16.727538 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:16.728741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:16.728741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:16.728741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:16.733332 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 618/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:39:16.734299 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15e52-2a2a-42ff-b878-ba6528cb0ca7 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:16.852847 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:16.859947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15e52-2a2a-42ff-b878-ba6528cb0ca7 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:16.900630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638526cb-8e5b-4743-b64d-5005ec1e2655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:16.901378 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638526cb-8e5b-4743-b64d-5005ec1e2655 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:16.901784 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638526cb-8e5b-4743-b64d-5005ec1e2655 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:16.902214 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638526cb-8e5b-4743-b64d-5005ec1e2655 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:16.903059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638526cb-8e5b-4743-b64d-5005ec1e2655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:16.903489 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638526cb-8e5b-4743-b64d-5005ec1e2655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:16.905764 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638526cb-8e5b-4743-b64d-5005ec1e2655 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:16.924153 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 625/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:16.930793 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5dccd-6621-4b45-9645-09133755a28b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:17.033528 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:17.065330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5dccd-6621-4b45-9645-09133755a28b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:17.107769 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fb150-ed82-4fff-8382-8719c797fce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:17.108613 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fb150-ed82-4fff-8382-8719c797fce3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:17.108962 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fb150-ed82-4fff-8382-8719c797fce3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:17.109383 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fb150-ed82-4fff-8382-8719c797fce3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:17.119296 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fb150-ed82-4fff-8382-8719c797fce3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:17.120883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fb150-ed82-4fff-8382-8719c797fce3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-1d0fb150-ed82-4fff-8382-8719c797fce3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:39:17.133725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:17.134922 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:17.135912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:39:17.135912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:39:17.135912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:39:17.135912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:39:17.135912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:39:17.135912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:39:17.135912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:39:17.135912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:17.135912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 619/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:39:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:17.142482 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638526cb-8e5b-4743-b64d-5005ec1e2655 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:17.272133 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638526cb-8e5b-4743-b64d-5005ec1e2655 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:17.312846 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11629141-c7e4-4c93-983f-07eda30e92e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:17.313474 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11629141-c7e4-4c93-983f-07eda30e92e2 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:17.313748 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11629141-c7e4-4c93-983f-07eda30e92e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:17.314062 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11629141-c7e4-4c93-983f-07eda30e92e2 None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:17.314760 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11629141-c7e4-4c93-983f-07eda30e92e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:17.315059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11629141-c7e4-4c93-983f-07eda30e92e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:17.316497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11629141-c7e4-4c93-983f-07eda30e92e2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:17.330365 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 626/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:39:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:17.341675 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fb150-ed82-4fff-8382-8719c797fce3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:17.472402 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fb150-ed82-4fff-8382-8719c797fce3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:17.518781 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f960d-0ad2-4663-bec1-6dc60030f3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:17.519574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f960d-0ad2-4663-bec1-6dc60030f3f4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:17.519975 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f960d-0ad2-4663-bec1-6dc60030f3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:17.520252 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f960d-0ad2-4663-bec1-6dc60030f3f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:17.521023 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f960d-0ad2-4663-bec1-6dc60030f3f4 None admin] RBAC: Adding query filter params (role.id=943fc0edf23046a69ba6102f2c1e544c, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=3a442a785ebe45709bce78add0e0e93a) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:17.521527 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f960d-0ad2-4663-bec1-6dc60030f3f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:17.523403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f960d-0ad2-4663-bec1-6dc60030f3f4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:17.530998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 620/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:39:17 2022] GET /identity/v3/role_assignments?user.id=3a442a785ebe45709bce78add0e0e93a&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=943fc0edf23046a69ba6102f2c1e544c => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:19.988358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da918479-93e9-46dd-9a2c-6aca48f129c6 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:19.988952 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da918479-93e9-46dd-9a2c-6aca48f129c6 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:19.988952 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da918479-93e9-46dd-9a2c-6aca48f129c6 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:19.990695 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 627/1247] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:39:19.996432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb722d7e-f3c4-490b-9d85-5ea3bd2ae7b8 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:19.996967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb722d7e-f3c4-490b-9d85-5ea3bd2ae7b8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:19.996967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb722d7e-f3c4-490b-9d85-5ea3bd2ae7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:20.003676 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:20.017721 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-bb722d7e-f3c4-490b-9d85-5ea3bd2ae7b8 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:39:20.018973 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bb722d7e-f3c4-490b-9d85-5ea3bd2ae7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:39:20.099732 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 621/1248] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:39:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:39:20.376962 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da918479-93e9-46dd-9a2c-6aca48f129c6 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:20.396666 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:20.485728 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da918479-93e9-46dd-9a2c-6aca48f129c6 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:20.523115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02615b48-69ac-4dae-b6e6-b054c1a58d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:20.523923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02615b48-69ac-4dae-b6e6-b054c1a58d85 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:20.523923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02615b48-69ac-4dae-b6e6-b054c1a58d85 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:20.524179 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02615b48-69ac-4dae-b6e6-b054c1a58d85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:20.535142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02615b48-69ac-4dae-b6e6-b054c1a58d85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-02615b48-69ac-4dae-b6e6-b054c1a58d85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:39:20.539970 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:20.541247 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:20.542338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 628/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:39:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:20.547402 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb722d7e-f3c4-490b-9d85-5ea3bd2ae7b8 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:20.659696 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb722d7e-f3c4-490b-9d85-5ea3bd2ae7b8 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:20.704685 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:20.710891 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf5dec-25a5-4f0c-8104-b0ff6f5790d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:20.711570 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf5dec-25a5-4f0c-8104-b0ff6f5790d9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:20.712121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf5dec-25a5-4f0c-8104-b0ff6f5790d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:20.712640 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf5dec-25a5-4f0c-8104-b0ff6f5790d9 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:20.714851 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf5dec-25a5-4f0c-8104-b0ff6f5790d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:20.715203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf5dec-25a5-4f0c-8104-b0ff6f5790d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:20.716591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf5dec-25a5-4f0c-8104-b0ff6f5790d9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:20.725384 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 622/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:39:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:20.731336 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02615b48-69ac-4dae-b6e6-b054c1a58d85 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:20.816217 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:20.844433 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02615b48-69ac-4dae-b6e6-b054c1a58d85 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:20.885793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601e501-9d11-46b3-b7d7-8003a581994d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:20.886648 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601e501-9d11-46b3-b7d7-8003a581994d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:20.887009 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601e501-9d11-46b3-b7d7-8003a581994d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:20.887333 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601e501-9d11-46b3-b7d7-8003a581994d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:20.900001 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601e501-9d11-46b3-b7d7-8003a581994d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:20.901505 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601e501-9d11-46b3-b7d7-8003a581994d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-0601e501-9d11-46b3-b7d7-8003a581994d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:20.918500 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:20.919624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:20.920873 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:20.920873 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:20.920873 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:20.925163 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 629/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:39:20.926504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf5dec-25a5-4f0c-8104-b0ff6f5790d9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:21.065547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf5dec-25a5-4f0c-8104-b0ff6f5790d9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:21.106716 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe7de2-ab33-4588-969d-e0e905d2302d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:21.107276 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe7de2-ab33-4588-969d-e0e905d2302d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:21.107539 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe7de2-ab33-4588-969d-e0e905d2302d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:21.107810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe7de2-ab33-4588-969d-e0e905d2302d None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:21.108466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe7de2-ab33-4588-969d-e0e905d2302d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:21.108874 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe7de2-ab33-4588-969d-e0e905d2302d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:21.110663 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe7de2-ab33-4588-969d-e0e905d2302d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:21.125281 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 623/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:39:21.131239 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601e501-9d11-46b3-b7d7-8003a581994d None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:21.257305 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601e501-9d11-46b3-b7d7-8003a581994d None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:21.288149 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:21.297328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2625cf8-4838-4a14-8e30-6a71608fdc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:21.297945 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2625cf8-4838-4a14-8e30-6a71608fdc53 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:21.298890 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2625cf8-4838-4a14-8e30-6a71608fdc53 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:21.299188 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2625cf8-4838-4a14-8e30-6a71608fdc53 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:21.307579 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2625cf8-4838-4a14-8e30-6a71608fdc53 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:21.309060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2625cf8-4838-4a14-8e30-6a71608fdc53 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-c2625cf8-4838-4a14-8e30-6a71608fdc53 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:39:21.321887 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:21.326137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:21.327503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:21.327503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:21.327503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:39:21.327503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:39:21.327503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:39:21.327503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:39:21.327503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:39:21.327503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:39:21.327503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 12:39:21.327503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:21.327503 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 630/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:21.329936 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe7de2-ab33-4588-969d-e0e905d2302d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:21.396418 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:21.451759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe7de2-ab33-4588-969d-e0e905d2302d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:21.492242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f0acd-0690-4e45-85bf-40c12e5beda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:21.492979 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f0acd-0690-4e45-85bf-40c12e5beda3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:21.493277 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f0acd-0690-4e45-85bf-40c12e5beda3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:21.493653 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f0acd-0690-4e45-85bf-40c12e5beda3 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:21.494598 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f0acd-0690-4e45-85bf-40c12e5beda3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:21.494902 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f0acd-0690-4e45-85bf-40c12e5beda3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:21.496524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f0acd-0690-4e45-85bf-40c12e5beda3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:21.517909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 624/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Tue Oct 4 12:39:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:21.525239 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2625cf8-4838-4a14-8e30-6a71608fdc53 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:21.633447 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2625cf8-4838-4a14-8e30-6a71608fdc53 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:21.670236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf52811-45b7-4481-8d7e-0a9afe138da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:21.670768 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf52811-45b7-4481-8d7e-0a9afe138da9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:21.671048 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf52811-45b7-4481-8d7e-0a9afe138da9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:21.671319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf52811-45b7-4481-8d7e-0a9afe138da9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:21.732449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:21.745945 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf52811-45b7-4481-8d7e-0a9afe138da9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:21.747460 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf52811-45b7-4481-8d7e-0a9afe138da9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:21.855489 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 631/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:39:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:39:21.861629 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f0acd-0690-4e45-85bf-40c12e5beda3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:21.995088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f0acd-0690-4e45-85bf-40c12e5beda3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:22.009011 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:22.033818 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd1b82-dafe-4873-a128-8fdf4dc40b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:22.034491 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd1b82-dafe-4873-a128-8fdf4dc40b9b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:22.034767 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd1b82-dafe-4873-a128-8fdf4dc40b9b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:22.035038 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd1b82-dafe-4873-a128-8fdf4dc40b9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:22.049503 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd1b82-dafe-4873-a128-8fdf4dc40b9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:22.051014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd1b82-dafe-4873-a128-8fdf4dc40b9b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-97fd1b82-dafe-4873-a128-8fdf4dc40b9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:22.066323 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:22.067847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:22.068750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:22.068750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:22.068750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:22.068750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 625/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 12:39:22.073287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf52811-45b7-4481-8d7e-0a9afe138da9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:22.166004 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:22.179296 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf52811-45b7-4481-8d7e-0a9afe138da9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:22.213248 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b6b7f-29d1-4a63-a4c7-9948f2562655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:22.213845 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b6b7f-29d1-4a63-a4c7-9948f2562655 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:22.214195 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b6b7f-29d1-4a63-a4c7-9948f2562655 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:22.214465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b6b7f-29d1-4a63-a4c7-9948f2562655 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:22.215090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b6b7f-29d1-4a63-a4c7-9948f2562655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:22.215390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b6b7f-29d1-4a63-a4c7-9948f2562655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:22.216762 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b6b7f-29d1-4a63-a4c7-9948f2562655 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:22.228267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 632/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:22.233967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd1b82-dafe-4873-a128-8fdf4dc40b9b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:22.336453 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd1b82-dafe-4873-a128-8fdf4dc40b9b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:22.369455 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e724e-5175-4698-b479-33d5f3fd955f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:22.370105 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e724e-5175-4698-b479-33d5f3fd955f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:22.370388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e724e-5175-4698-b479-33d5f3fd955f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:22.370662 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e724e-5175-4698-b479-33d5f3fd955f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:22.377111 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e724e-5175-4698-b479-33d5f3fd955f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:22.378583 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e724e-5175-4698-b479-33d5f3fd955f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-ec3e724e-5175-4698-b479-33d5f3fd955f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:39:22.390596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:22.391620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:22.393362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:39:22.393362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:39:22.393362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:39:22.393362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:39:22.393362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:39:22.393362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:39:22.393362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:39:22.393362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:22.393362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 626/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:39:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:22.398125 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b6b7f-29d1-4a63-a4c7-9948f2562655 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:22.496241 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b6b7f-29d1-4a63-a4c7-9948f2562655 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:22.528298 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274bdd6-5f1c-4527-b8ef-083c9464e56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:22.528828 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274bdd6-5f1c-4527-b8ef-083c9464e56d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:22.529106 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274bdd6-5f1c-4527-b8ef-083c9464e56d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:22.529372 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274bdd6-5f1c-4527-b8ef-083c9464e56d None admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:22.530066 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274bdd6-5f1c-4527-b8ef-083c9464e56d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:22.530367 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274bdd6-5f1c-4527-b8ef-083c9464e56d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:22.531779 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274bdd6-5f1c-4527-b8ef-083c9464e56d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:22.543749 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 633/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:39:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:22.550352 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e724e-5175-4698-b479-33d5f3fd955f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:22.575104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:22.647098 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e724e-5175-4698-b479-33d5f3fd955f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:22.679509 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fc2dc-0e62-4cd5-9697-1bd8249ecd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:22.680104 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fc2dc-0e62-4cd5-9697-1bd8249ecd85 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:22.680382 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fc2dc-0e62-4cd5-9697-1bd8249ecd85 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:22.680646 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fc2dc-0e62-4cd5-9697-1bd8249ecd85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:22.681359 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fc2dc-0e62-4cd5-9697-1bd8249ecd85 None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=3a442a785ebe45709bce78add0e0e93a) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:22.681659 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fc2dc-0e62-4cd5-9697-1bd8249ecd85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:22.683103 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fc2dc-0e62-4cd5-9697-1bd8249ecd85 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:22.688273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 627/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:39:22 2022] GET /identity/v3/role_assignments?user.id=3a442a785ebe45709bce78add0e0e93a&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=fc68d072459d4720b9666c93bd535d63 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:24.858448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e101091-98fe-4416-bb05-603d10cd4711 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:24.858981 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e101091-98fe-4416-bb05-603d10cd4711 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:24.859074 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e101091-98fe-4416-bb05-603d10cd4711 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:24.860338 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 634/1261] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:39:24.865360 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becbca2-7612-469a-8a31-f2908704332c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:24.865744 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becbca2-7612-469a-8a31-f2908704332c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:24.865907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becbca2-7612-469a-8a31-f2908704332c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:24.886833 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5becbca2-7612-469a-8a31-f2908704332c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:39:24.888363 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5becbca2-7612-469a-8a31-f2908704332c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:39:24.972246 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 628/1262] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:39:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:39:25.240489 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e101091-98fe-4416-bb05-603d10cd4711 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:25.251730 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:25.343083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e101091-98fe-4416-bb05-603d10cd4711 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:25.375899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e8683-2c94-4f46-ae33-4dc1009ae102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:25.377059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e8683-2c94-4f46-ae33-4dc1009ae102 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:25.377335 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e8683-2c94-4f46-ae33-4dc1009ae102 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:25.377603 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e8683-2c94-4f46-ae33-4dc1009ae102 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:25.387390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e8683-2c94-4f46-ae33-4dc1009ae102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-077e8683-2c94-4f46-ae33-4dc1009ae102 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:39:25.392381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:25.393698 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:25.394986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 635/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:39:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:25.399521 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becbca2-7612-469a-8a31-f2908704332c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:25.420544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:25.499521 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becbca2-7612-469a-8a31-f2908704332c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:25.533018 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1deaf-57ae-458b-b632-273b262fa485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:25.533617 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1deaf-57ae-458b-b632-273b262fa485 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:25.533885 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1deaf-57ae-458b-b632-273b262fa485 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:25.534180 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1deaf-57ae-458b-b632-273b262fa485 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:25.534828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1deaf-57ae-458b-b632-273b262fa485 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:25.535122 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1deaf-57ae-458b-b632-273b262fa485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:25.536472 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1deaf-57ae-458b-b632-273b262fa485 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:25.544321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 629/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:39:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:25.550777 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e8683-2c94-4f46-ae33-4dc1009ae102 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:25.617775 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:25.650723 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e8683-2c94-4f46-ae33-4dc1009ae102 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:25.684794 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c0d56-0289-49e6-9c22-55b900205e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:25.685403 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c0d56-0289-49e6-9c22-55b900205e3d None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:25.685676 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c0d56-0289-49e6-9c22-55b900205e3d None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:25.685941 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c0d56-0289-49e6-9c22-55b900205e3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:25.695707 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c0d56-0289-49e6-9c22-55b900205e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:25.697141 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c0d56-0289-49e6-9c22-55b900205e3d None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-ff1c0d56-0289-49e6-9c22-55b900205e3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:25.711180 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:25.712293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:25.713408 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:25.713408 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:25.713408 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:25.713408 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 636/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:25.718203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1deaf-57ae-458b-b632-273b262fa485 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:25.801569 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:25.821520 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1deaf-57ae-458b-b632-273b262fa485 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:25.856184 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbe351-535f-4310-9f3c-8263e4036955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:25.856787 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbe351-535f-4310-9f3c-8263e4036955 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:25.857063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbe351-535f-4310-9f3c-8263e4036955 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:25.857332 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbe351-535f-4310-9f3c-8263e4036955 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:25.857966 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbe351-535f-4310-9f3c-8263e4036955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:25.858294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbe351-535f-4310-9f3c-8263e4036955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:25.859648 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbe351-535f-4310-9f3c-8263e4036955 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:25.871140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 630/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:25.876992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c0d56-0289-49e6-9c22-55b900205e3d None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:25.981820 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c0d56-0289-49e6-9c22-55b900205e3d None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:26.000977 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:26.015520 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d71fe-1e0f-48d3-b401-83c0e92c13af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:26.016138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d71fe-1e0f-48d3-b401-83c0e92c13af None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:26.016415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d71fe-1e0f-48d3-b401-83c0e92c13af None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:26.017219 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d71fe-1e0f-48d3-b401-83c0e92c13af None admin] PATH_INFO: `/v3/users/placement` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:26.024614 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d71fe-1e0f-48d3-b401-83c0e92c13af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:26.026059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d71fe-1e0f-48d3-b401-83c0e92c13af None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-2e5d71fe-1e0f-48d3-b401-83c0e92c13af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:39:26.037371 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:26.038532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:26.039718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:26.039718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:26.039718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:39:26.039718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:39:26.039718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:39:26.039718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:39:26.039718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:39:26.039718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:39:26.039718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 12:39:26.039718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:26.039718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 637/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:26.044551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbe351-535f-4310-9f3c-8263e4036955 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:26.142464 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbe351-535f-4310-9f3c-8263e4036955 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:26.175273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd23fff-c71f-44e6-9662-06f080eae35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:26.175868 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd23fff-c71f-44e6-9662-06f080eae35c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:26.176140 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd23fff-c71f-44e6-9662-06f080eae35c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:26.176406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd23fff-c71f-44e6-9662-06f080eae35c None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:26.177123 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd23fff-c71f-44e6-9662-06f080eae35c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:26.177427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd23fff-c71f-44e6-9662-06f080eae35c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:26.178863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd23fff-c71f-44e6-9662-06f080eae35c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:26.187260 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:26.196951 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 631/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Tue Oct 4 12:39:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:26.202946 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d71fe-1e0f-48d3-b401-83c0e92c13af None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:26.301200 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d71fe-1e0f-48d3-b401-83c0e92c13af None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:26.334675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4be080-f514-4db3-a14f-ba2781e4ea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:26.335292 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4be080-f514-4db3-a14f-ba2781e4ea22 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:26.335562 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4be080-f514-4db3-a14f-ba2781e4ea22 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:26.335830 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4be080-f514-4db3-a14f-ba2781e4ea22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:26.346233 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4be080-f514-4db3-a14f-ba2781e4ea22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:26.347674 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4be080-f514-4db3-a14f-ba2781e4ea22 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-8d4be080-f514-4db3-a14f-ba2781e4ea22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:26.361362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:26.362539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:26.363661 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:26.363661 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:26.363661 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:26.363661 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 638/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:26.368306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd23fff-c71f-44e6-9662-06f080eae35c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:26.471501 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd23fff-c71f-44e6-9662-06f080eae35c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:26.505231 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0e2ec-de5f-46bb-8abe-feac4720c3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:26.505834 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0e2ec-de5f-46bb-8abe-feac4720c3e6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:26.506131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0e2ec-de5f-46bb-8abe-feac4720c3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:26.506398 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0e2ec-de5f-46bb-8abe-feac4720c3e6 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:26.507026 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0e2ec-de5f-46bb-8abe-feac4720c3e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:26.507322 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0e2ec-de5f-46bb-8abe-feac4720c3e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:26.508678 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0e2ec-de5f-46bb-8abe-feac4720c3e6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:26.520449 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 632/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:26.526279 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4be080-f514-4db3-a14f-ba2781e4ea22 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:26.557429 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:26.632610 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4be080-f514-4db3-a14f-ba2781e4ea22 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:26.666020 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1929ed8-08ba-46a4-9f36-d7fb1b1e3e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:26.666664 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1929ed8-08ba-46a4-9f36-d7fb1b1e3e93 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:26.667193 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1929ed8-08ba-46a4-9f36-d7fb1b1e3e93 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:26.667193 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1929ed8-08ba-46a4-9f36-d7fb1b1e3e93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:26.674648 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1929ed8-08ba-46a4-9f36-d7fb1b1e3e93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:26.676087 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1929ed8-08ba-46a4-9f36-d7fb1b1e3e93 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-d1929ed8-08ba-46a4-9f36-d7fb1b1e3e93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:39:26.687202 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:26.688325 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:26.689875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:39:26.689875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:39:26.689875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:39:26.689875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:39:26.689875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:39:26.689875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:39:26.689875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:39:26.689875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:26.689875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 639/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:39:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:26.694883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0e2ec-de5f-46bb-8abe-feac4720c3e6 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:26.752555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:26.793500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0e2ec-de5f-46bb-8abe-feac4720c3e6 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:26.826911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb768e-6d3b-4f8f-85ea-25082c13ce06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:26.827518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb768e-6d3b-4f8f-85ea-25082c13ce06 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:26.827790 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb768e-6d3b-4f8f-85ea-25082c13ce06 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:26.828059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb768e-6d3b-4f8f-85ea-25082c13ce06 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:26.828739 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb768e-6d3b-4f8f-85ea-25082c13ce06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:26.829039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb768e-6d3b-4f8f-85ea-25082c13ce06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:26.830528 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb768e-6d3b-4f8f-85ea-25082c13ce06 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:26.842294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 633/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:39:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:26.848353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1929ed8-08ba-46a4-9f36-d7fb1b1e3e93 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:26.952864 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:26.962118 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1929ed8-08ba-46a4-9f36-d7fb1b1e3e93 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:26.997519 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4eaa4-c175-462c-bccc-9457ba1dcdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:26.998287 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4eaa4-c175-462c-bccc-9457ba1dcdc2 None admin] REQUEST_METHOD: `PUT` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:26.998567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4eaa4-c175-462c-bccc-9457ba1dcdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:26.998840 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4eaa4-c175-462c-bccc-9457ba1dcdc2 None admin] PATH_INFO: `/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/3a442a785ebe45709bce78add0e0e93a/roles/fc68d072459d4720b9666c93bd535d63` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:27.000789 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4eaa4-c175-462c-bccc-9457ba1dcdc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a7fb003a695490faa0077bbbe5e7d77, user_id=3a442a785ebe45709bce78add0e0e93a, role_id=fc68d072459d4720b9666c93bd535d63)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:27.002313 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4eaa4-c175-462c-bccc-9457ba1dcdc2 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:27.016496 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 640/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Tue Oct 4 12:39:26 2022] PUT /identity/v3/projects/6a7fb003a695490faa0077bbbe5e7d77/users/3a442a785ebe45709bce78add0e0e93a/roles/fc68d072459d4720b9666c93bd535d63 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:39:29.141567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d5a4-6b92-4276-b705-6c3f24cac094 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:29.142126 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d5a4-6b92-4276-b705-6c3f24cac094 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:29.142126 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d5a4-6b92-4276-b705-6c3f24cac094 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:29.143463 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 634/1274] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:39:29.150373 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1064a0-1092-4e65-b910-9ea21540d328 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:29.150826 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1064a0-1092-4e65-b910-9ea21540d328 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:29.150932 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1064a0-1092-4e65-b910-9ea21540d328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:29.172495 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-bc1064a0-1092-4e65-b910-9ea21540d328 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:39:29.174285 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-bc1064a0-1092-4e65-b910-9ea21540d328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:39:29.238544 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 641/1275] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:39:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:39:29.507131 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487d5a4-6b92-4276-b705-6c3f24cac094 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:29.614422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487d5a4-6b92-4276-b705-6c3f24cac094 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:29.631256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:29.648872 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67ae3b-54e8-453c-a05e-4927c23f4a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:29.649495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67ae3b-54e8-453c-a05e-4927c23f4a6d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:29.649768 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67ae3b-54e8-453c-a05e-4927c23f4a6d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:29.650056 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67ae3b-54e8-453c-a05e-4927c23f4a6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:29.660521 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67ae3b-54e8-453c-a05e-4927c23f4a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-2c67ae3b-54e8-453c-a05e-4927c23f4a6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 12:39:29.665256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:29.666451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:29.667313 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 635/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Tue Oct 4 12:39:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:29.672411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1064a0-1092-4e65-b910-9ea21540d328 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:29.752142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:29.772106 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1064a0-1092-4e65-b910-9ea21540d328 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:29.808385 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23392cc0-5f6b-4982-93b8-1720bf38a954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:29.808979 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23392cc0-5f6b-4982-93b8-1720bf38a954 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:29.809250 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23392cc0-5f6b-4982-93b8-1720bf38a954 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:29.809518 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23392cc0-5f6b-4982-93b8-1720bf38a954 None admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:29.810200 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23392cc0-5f6b-4982-93b8-1720bf38a954 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:29.810497 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23392cc0-5f6b-4982-93b8-1720bf38a954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:29.811853 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23392cc0-5f6b-4982-93b8-1720bf38a954 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:29.819864 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 642/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Tue Oct 4 12:39:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:29.825798 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67ae3b-54e8-453c-a05e-4927c23f4a6d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:29.942942 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67ae3b-54e8-453c-a05e-4927c23f4a6d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:29.983954 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c889e78-30ab-497a-88e5-cc783f3e02f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:29.984591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c889e78-30ab-497a-88e5-cc783f3e02f0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:29.984866 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c889e78-30ab-497a-88e5-cc783f3e02f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:29.985136 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c889e78-30ab-497a-88e5-cc783f3e02f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:29.996127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c889e78-30ab-497a-88e5-cc783f3e02f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:29.997596 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c889e78-30ab-497a-88e5-cc783f3e02f0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-9c889e78-30ab-497a-88e5-cc783f3e02f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:30.013489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:30.015001 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:30.016097 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:30.016097 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:30.016097 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:30.016097 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 636/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:30.021770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23392cc0-5f6b-4982-93b8-1720bf38a954 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:30.121803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23392cc0-5f6b-4982-93b8-1720bf38a954 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:30.158355 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93a75d-eed2-4405-b55f-0347ee226fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:30.159015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93a75d-eed2-4405-b55f-0347ee226fb7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:30.159213 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93a75d-eed2-4405-b55f-0347ee226fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:30.159478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93a75d-eed2-4405-b55f-0347ee226fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:30.160102 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93a75d-eed2-4405-b55f-0347ee226fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:30.160391 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93a75d-eed2-4405-b55f-0347ee226fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:30.161752 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93a75d-eed2-4405-b55f-0347ee226fb7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:30.173619 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 643/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:30.179567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c889e78-30ab-497a-88e5-cc783f3e02f0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:30.212055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:30.286961 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c889e78-30ab-497a-88e5-cc783f3e02f0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:30.322386 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cb963-0eeb-4a1b-839c-866046a2b4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:30.323022 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cb963-0eeb-4a1b-839c-866046a2b4db None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:30.323298 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cb963-0eeb-4a1b-839c-866046a2b4db None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:30.323567 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cb963-0eeb-4a1b-839c-866046a2b4db None admin] PATH_INFO: `/v3/users/placement` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:30.331190 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cb963-0eeb-4a1b-839c-866046a2b4db None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:30.332637 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cb963-0eeb-4a1b-839c-866046a2b4db None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-9c1cb963-0eeb-4a1b-839c-866046a2b4db None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 12:39:30.343745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:30.344886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:30.346262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:30.346262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:30.346262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 12:39:30.346262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 12:39:30.346262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 12:39:30.346262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 12:39:30.346262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 12:39:30.346262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 12:39:30.346262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 12:39:30.346262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:30.346262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 637/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:30.351037 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93a75d-eed2-4405-b55f-0347ee226fb7 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:30.354871 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:30.453502 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93a75d-eed2-4405-b55f-0347ee226fb7 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:30.487869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888b4c3-b69e-4e56-8002-64e9dac72200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:30.488467 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888b4c3-b69e-4e56-8002-64e9dac72200 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:30.488740 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888b4c3-b69e-4e56-8002-64e9dac72200 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:30.489009 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888b4c3-b69e-4e56-8002-64e9dac72200 None admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:30.489728 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888b4c3-b69e-4e56-8002-64e9dac72200 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:30.490030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888b4c3-b69e-4e56-8002-64e9dac72200 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:30.491492 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888b4c3-b69e-4e56-8002-64e9dac72200 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:30.510370 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 644/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Tue Oct 4 12:39:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:30.516647 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cb963-0eeb-4a1b-839c-866046a2b4db None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:30.614795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:30.624541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cb963-0eeb-4a1b-839c-866046a2b4db None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:30.659280 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d22379-41ab-4c4e-bdf4-d15863a002d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:30.659884 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d22379-41ab-4c4e-bdf4-d15863a002d0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:30.660159 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d22379-41ab-4c4e-bdf4-d15863a002d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:30.660428 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d22379-41ab-4c4e-bdf4-d15863a002d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:30.727529 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d22379-41ab-4c4e-bdf4-d15863a002d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:30.728898 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d22379-41ab-4c4e-bdf4-d15863a002d0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:30.819863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:30.830808 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 638/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Tue Oct 4 12:39:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 12:39:30.836855 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888b4c3-b69e-4e56-8002-64e9dac72200 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:30.900940 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:30.944088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888b4c3-b69e-4e56-8002-64e9dac72200 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:30.981773 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7677dd-c800-422b-b0a0-bf5b8485c6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:30.982353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7677dd-c800-422b-b0a0-bf5b8485c6ee None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:30.982629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7677dd-c800-422b-b0a0-bf5b8485c6ee None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:30.982900 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7677dd-c800-422b-b0a0-bf5b8485c6ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:30.994369 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7677dd-c800-422b-b0a0-bf5b8485c6ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:30.995853 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7677dd-c800-422b-b0a0-bf5b8485c6ee None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-ce7677dd-c800-422b-b0a0-bf5b8485c6ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 12:39:31.011181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:31.012992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 12:39:31.014280 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 12:39:31.014280 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 12:39:31.014280 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:31.014280 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 645/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:31.020410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d22379-41ab-4c4e-bdf4-d15863a002d0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:31.123563 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d22379-41ab-4c4e-bdf4-d15863a002d0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:31.158060 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c036d13-b627-4768-bbd3-32ff96b79bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:31.158729 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c036d13-b627-4768-bbd3-32ff96b79bc1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:31.159003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c036d13-b627-4768-bbd3-32ff96b79bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:31.159271 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c036d13-b627-4768-bbd3-32ff96b79bc1 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:31.159912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c036d13-b627-4768-bbd3-32ff96b79bc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:31.160210 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c036d13-b627-4768-bbd3-32ff96b79bc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:31.161581 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c036d13-b627-4768-bbd3-32ff96b79bc1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:31.173329 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 639/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Tue Oct 4 12:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:39:31.179461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7677dd-c800-422b-b0a0-bf5b8485c6ee None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:31.281598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7677dd-c800-422b-b0a0-bf5b8485c6ee None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:31.296246 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:31.316660 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bae40-d347-4982-9385-b75a9bf4091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:31.317286 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bae40-d347-4982-9385-b75a9bf4091b None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:31.317565 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bae40-d347-4982-9385-b75a9bf4091b None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:31.317833 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bae40-d347-4982-9385-b75a9bf4091b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:31.324593 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68bae40-d347-4982-9385-b75a9bf4091b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:31.326073 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68bae40-d347-4982-9385-b75a9bf4091b None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application [None req-a68bae40-d347-4982-9385-b75a9bf4091b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 12:39:31.338587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:31.339751 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:31.340870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 12:39:31.340870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 12:39:31.340870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 12:39:31.340870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 12:39:31.340870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 12:39:31.340870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 12:39:31.340870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 12:39:31.340870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: ERROR keystone.server.flask.application Oct 04 12:39:31.340870 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 646/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Tue Oct 4 12:39:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:31.346998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c036d13-b627-4768-bbd3-32ff96b79bc1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:31.397651 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:31.449752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c036d13-b627-4768-bbd3-32ff96b79bc1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:31.483588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12640510-818c-483f-8fb7-a6d11099aae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:31.484198 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12640510-818c-483f-8fb7-a6d11099aae0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:31.484473 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12640510-818c-483f-8fb7-a6d11099aae0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:31.484744 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12640510-818c-483f-8fb7-a6d11099aae0 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:31.485481 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12640510-818c-483f-8fb7-a6d11099aae0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:31.485785 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12640510-818c-483f-8fb7-a6d11099aae0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:31.487269 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12640510-818c-483f-8fb7-a6d11099aae0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:31.500635 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 640/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Tue Oct 4 12:39:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 12:39:31.507509 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bae40-d347-4982-9385-b75a9bf4091b None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:31.609079 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bae40-d347-4982-9385-b75a9bf4091b None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:31.647543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fff8f2-2219-4296-9dff-ce381e93efdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:31.648146 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fff8f2-2219-4296-9dff-ce381e93efdd None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:31.648428 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fff8f2-2219-4296-9dff-ce381e93efdd None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:31.648702 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fff8f2-2219-4296-9dff-ce381e93efdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:31.649427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fff8f2-2219-4296-9dff-ce381e93efdd None admin] RBAC: Adding query filter params (role.id=fc68d072459d4720b9666c93bd535d63, scope.project.id=6a7fb003a695490faa0077bbbe5e7d77, user.id=3a442a785ebe45709bce78add0e0e93a) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:31.649736 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fff8f2-2219-4296-9dff-ce381e93efdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:31.651212 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fff8f2-2219-4296-9dff-ce381e93efdd None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:31.657421 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 647/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Tue Oct 4 12:39:31 2022] GET /identity/v3/role_assignments?user.id=3a442a785ebe45709bce78add0e0e93a&scope.project.id=6a7fb003a695490faa0077bbbe5e7d77&role.id=fc68d072459d4720b9666c93bd535d63 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:33.877423 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689533b-6d72-49a9-85d5-0ad3d8ee731c None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:33.877990 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689533b-6d72-49a9-85d5-0ad3d8ee731c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:33.877990 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689533b-6d72-49a9-85d5-0ad3d8ee731c None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:33.879433 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 641/1288] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:39:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:39:33.885956 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a530e-2d89-44d8-ac34-c87d014308b7 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:33.887210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a530e-2d89-44d8-ac34-c87d014308b7 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:33.887210 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a530e-2d89-44d8-ac34-c87d014308b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:33.909324 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-933a530e-2d89-44d8-ac34-c87d014308b7 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:39:33.910711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-933a530e-2d89-44d8-ac34-c87d014308b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:39:33.949762 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:33.996452 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 648/1289] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:39:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:39:34.269379 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689533b-6d72-49a9-85d5-0ad3d8ee731c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:34.372132 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689533b-6d72-49a9-85d5-0ad3d8ee731c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:34.383287 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:34.408683 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb191a48-8642-4b9c-8423-4304f1b0951a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:34.409338 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb191a48-8642-4b9c-8423-4304f1b0951a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:34.409611 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb191a48-8642-4b9c-8423-4304f1b0951a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:34.409878 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb191a48-8642-4b9c-8423-4304f1b0951a None admin] PATH_INFO: `/v3/services/placement` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:34.410510 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb191a48-8642-4b9c-8423-4304f1b0951a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:34.411896 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb191a48-8642-4b9c-8423-4304f1b0951a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-fb191a48-8642-4b9c-8423-4304f1b0951a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:34.421413 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:34.423424 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:34.425283 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 642/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Tue Oct 4 12:39:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:34.430623 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a530e-2d89-44d8-ac34-c87d014308b7 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:34.536995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a530e-2d89-44d8-ac34-c87d014308b7 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:34.574193 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f5371-e42d-46be-9416-f8f242afe52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:34.574826 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f5371-e42d-46be-9416-f8f242afe52c None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:34.575099 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f5371-e42d-46be-9416-f8f242afe52c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:34.575363 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f5371-e42d-46be-9416-f8f242afe52c None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:34.576020 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f5371-e42d-46be-9416-f8f242afe52c None admin] RBAC: Adding query filter params (name=placement) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:34.576321 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f5371-e42d-46be-9416-f8f242afe52c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:34.577760 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f5371-e42d-46be-9416-f8f242afe52c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:34.584008 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 649/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Tue Oct 4 12:39:34 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:34.590369 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb191a48-8642-4b9c-8423-4304f1b0951a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:34.707285 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb191a48-8642-4b9c-8423-4304f1b0951a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:34.748200 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927cc67-227a-4587-8ea8-82702f94d454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:34.748826 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927cc67-227a-4587-8ea8-82702f94d454 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:34.749101 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927cc67-227a-4587-8ea8-82702f94d454 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:34.749374 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927cc67-227a-4587-8ea8-82702f94d454 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:34.750021 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927cc67-227a-4587-8ea8-82702f94d454 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:34.750353 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927cc67-227a-4587-8ea8-82702f94d454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:34.752318 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927cc67-227a-4587-8ea8-82702f94d454 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:34.757524 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 643/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Tue Oct 4 12:39:34 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:37.046167 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86ab3d-0abd-4024-8125-4ce0e45426db None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:37.046749 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86ab3d-0abd-4024-8125-4ce0e45426db None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:37.046749 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86ab3d-0abd-4024-8125-4ce0e45426db None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:37.048794 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 650/1293] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:39:37.055280 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f4118-a619-402a-b13a-d545649b3267 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:37.055810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f4118-a619-402a-b13a-d545649b3267 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:37.055810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f4118-a619-402a-b13a-d545649b3267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:37.080768 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b66f4118-a619-402a-b13a-d545649b3267 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:39:37.081994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b66f4118-a619-402a-b13a-d545649b3267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:39:37.114961 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:37.154132 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 644/1294] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:39:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:39:37.426262 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86ab3d-0abd-4024-8125-4ce0e45426db None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:37.477803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:37.554187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86ab3d-0abd-4024-8125-4ce0e45426db None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:37.592172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44660353-0317-4188-ac0e-f25ca5fdb24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:37.592765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44660353-0317-4188-ac0e-f25ca5fdb24c None admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:37.593043 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44660353-0317-4188-ac0e-f25ca5fdb24c None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:37.593341 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44660353-0317-4188-ac0e-f25ca5fdb24c None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:37.594083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44660353-0317-4188-ac0e-f25ca5fdb24c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:37.596046 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44660353-0317-4188-ac0e-f25ca5fdb24c None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:37.614627 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 651/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Tue Oct 4 12:39:37 2022] POST /identity/v3/services => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:39:39.859238 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320a62e-8c45-4d07-903d-58b8529d08c3 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:39.859821 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320a62e-8c45-4d07-903d-58b8529d08c3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:39.859821 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320a62e-8c45-4d07-903d-58b8529d08c3 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:39.861148 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 645/1296] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:39:39.866792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141ca11-bad8-4596-a690-0a73e5ab83c9 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:39.867261 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141ca11-bad8-4596-a690-0a73e5ab83c9 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:39.867359 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141ca11-bad8-4596-a690-0a73e5ab83c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:39.899332 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-3141ca11-bad8-4596-a690-0a73e5ab83c9 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:39:39.900769 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-3141ca11-bad8-4596-a690-0a73e5ab83c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:39:39.989598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 652/1297] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:39:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:39:40.261130 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320a62e-8c45-4d07-903d-58b8529d08c3 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:40.373779 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320a62e-8c45-4d07-903d-58b8529d08c3 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:40.415139 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09892d5f-b9a2-47c7-9f58-ec050c627913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:40.415809 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09892d5f-b9a2-47c7-9f58-ec050c627913 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:40.416086 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09892d5f-b9a2-47c7-9f58-ec050c627913 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:40.416355 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09892d5f-b9a2-47c7-9f58-ec050c627913 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:40.416995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09892d5f-b9a2-47c7-9f58-ec050c627913 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:40.418507 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09892d5f-b9a2-47c7-9f58-ec050c627913 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-09892d5f-b9a2-47c7-9f58-ec050c627913 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:40.428504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:40.429810 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:40.431106 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 646/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Tue Oct 4 12:39:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:40.439739 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141ca11-bad8-4596-a690-0a73e5ab83c9 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:40.464778 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:40.578267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141ca11-bad8-4596-a690-0a73e5ab83c9 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:40.614283 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78887632-660f-446a-a9e5-d906100609d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:40.614918 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78887632-660f-446a-a9e5-d906100609d9 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:40.615170 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78887632-660f-446a-a9e5-d906100609d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:40.615435 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78887632-660f-446a-a9e5-d906100609d9 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:40.616061 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78887632-660f-446a-a9e5-d906100609d9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:40.616358 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78887632-660f-446a-a9e5-d906100609d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:40.617735 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78887632-660f-446a-a9e5-d906100609d9 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:40.623493 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 653/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Tue Oct 4 12:39:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:40.630302 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09892d5f-b9a2-47c7-9f58-ec050c627913 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:40.666462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:40.744888 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09892d5f-b9a2-47c7-9f58-ec050c627913 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:40.781022 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412616c-f43c-4041-805b-cbcaab9643f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:40.781659 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412616c-f43c-4041-805b-cbcaab9643f4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:40.782094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412616c-f43c-4041-805b-cbcaab9643f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:40.782607 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412616c-f43c-4041-805b-cbcaab9643f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:40.783338 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412616c-f43c-4041-805b-cbcaab9643f4 None admin] RBAC: Adding query filter params (interface=public, service_id=934f9fdf697441949d61eb077d71c656, region_id=RegionOne) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:40.783650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412616c-f43c-4041-805b-cbcaab9643f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:40.785023 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412616c-f43c-4041-805b-cbcaab9643f4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:40.790449 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 647/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Tue Oct 4 12:39:40 2022] GET /identity/v3/endpoints?service_id=934f9fdf697441949d61eb077d71c656&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:40.796383 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78887632-660f-446a-a9e5-d906100609d9 None admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:40.896491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:40.915226 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78887632-660f-446a-a9e5-d906100609d9 None admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:40.952510 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf676e-cb88-487e-a18b-53cfaecaabd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:40.953148 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf676e-cb88-487e-a18b-53cfaecaabd3 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:40.953447 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf676e-cb88-487e-a18b-53cfaecaabd3 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:40.953727 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf676e-cb88-487e-a18b-53cfaecaabd3 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:40.954345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf676e-cb88-487e-a18b-53cfaecaabd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:40.955719 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf676e-cb88-487e-a18b-53cfaecaabd3 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:40.963134 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 654/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Tue Oct 4 12:39:40 2022] GET /identity/v3/services => generated 2191 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:39:43.196206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f15d7c-51af-4bce-9b17-cee9fdd20c46 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:43.196786 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f15d7c-51af-4bce-9b17-cee9fdd20c46 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:43.196885 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f15d7c-51af-4bce-9b17-cee9fdd20c46 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:43.198180 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 648/1302] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:39:43.204100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b4d8f-da4c-4705-aef1-e5930830e5fa None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:43.204630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b4d8f-da4c-4705-aef1-e5930830e5fa None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:43.204630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b4d8f-da4c-4705-aef1-e5930830e5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:43.224533 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-e03b4d8f-da4c-4705-aef1-e5930830e5fa None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:39:43.226267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-e03b4d8f-da4c-4705-aef1-e5930830e5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:39:43.300390 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 655/1303] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:39:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 12:39:43.572108 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f15d7c-51af-4bce-9b17-cee9fdd20c46 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:43.683400 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:43.691511 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f15d7c-51af-4bce-9b17-cee9fdd20c46 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:43.730875 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ff10a-9d42-4d3a-b36f-022877f115d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:43.731533 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ff10a-9d42-4d3a-b36f-022877f115d8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:43.731809 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ff10a-9d42-4d3a-b36f-022877f115d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:43.732081 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ff10a-9d42-4d3a-b36f-022877f115d8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:43.732683 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ff10a-9d42-4d3a-b36f-022877f115d8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:43.734162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ff10a-9d42-4d3a-b36f-022877f115d8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-bd5ff10a-9d42-4d3a-b36f-022877f115d8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 12:39:43.744540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 12:39:43.745850 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Oct 04 12:39:43.746914 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 649/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Tue Oct 4 12:39:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 12:39:43.752335 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b4d8f-da4c-4705-aef1-e5930830e5fa None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:43.839424 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:43.865814 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b4d8f-da4c-4705-aef1-e5930830e5fa None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:43.903205 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cb43f-0735-40ac-b054-c1b2ee5f84d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:43.903819 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cb43f-0735-40ac-b054-c1b2ee5f84d7 None admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:43.904097 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cb43f-0735-40ac-b054-c1b2ee5f84d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:43.904368 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cb43f-0735-40ac-b054-c1b2ee5f84d7 None admin] PATH_INFO: `/v3/services` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:43.905010 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cb43f-0735-40ac-b054-c1b2ee5f84d7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:39:43.905311 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cb43f-0735-40ac-b054-c1b2ee5f84d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:43.907344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cb43f-0735-40ac-b054-c1b2ee5f84d7 None admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:43.913257 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 656/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Tue Oct 4 12:39:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:39:43.919428 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ff10a-9d42-4d3a-b36f-022877f115d8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:39:44.036488 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ff10a-9d42-4d3a-b36f-022877f115d8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:39:44.073711 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773ab37-9d92-4211-9051-9e2e1413b0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:39:44.074347 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773ab37-9d92-4211-9051-9e2e1413b0af None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:39:44.074628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773ab37-9d92-4211-9051-9e2e1413b0af None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:39:44.074896 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773ab37-9d92-4211-9051-9e2e1413b0af None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:39:44.075560 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773ab37-9d92-4211-9051-9e2e1413b0af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:39:44.076916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773ab37-9d92-4211-9051-9e2e1413b0af None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:39:44.097571 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 650/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Tue Oct 4 12:39:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:39:53.910631 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=30 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:39:54.092559 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:40:50.277267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42974f7-1721-4b1c-b74e-896e49dedf67 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:40:50.277992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42974f7-1721-4b1c-b74e-896e49dedf67 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:40:50.277992 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42974f7-1721-4b1c-b74e-896e49dedf67 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:40:50.280212 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 657/1307] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:40:50.290262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff31e4-1f02-48d5-b6f3-ac3793b52156 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:40:50.290725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff31e4-1f02-48d5-b6f3-ac3793b52156 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:40:50.291280 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff31e4-1f02-48d5-b6f3-ac3793b52156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:40:50.470214 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-d5ff31e4-1f02-48d5-b6f3-ac3793b52156 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:40:50.472063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d5ff31e4-1f02-48d5-b6f3-ac3793b52156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:40:50.696183 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 651/1308] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:40:51.017280 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c7ad8-9c9a-48aa-8e6c-40fb927e7ee3 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:40:51.017941 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c7ad8-9c9a-48aa-8e6c-40fb927e7ee3 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:40:51.018468 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c7ad8-9c9a-48aa-8e6c-40fb927e7ee3 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:40:51.020439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 658/1309] 10.222.0.33 () {54 vars in 964 bytes} [Tue Oct 4 12:40:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:40:51.027469 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa0c99-2eb8-41fa-ac1b-769e25c16d9f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:40:51.027842 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa0c99-2eb8-41fa-ac1b-769e25c16d9f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:40:51.028622 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa0c99-2eb8-41fa-ac1b-769e25c16d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:40:51.131943 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a3fa0c99-2eb8-41fa-ac1b-769e25c16d9f None None] MFA Rules not processed for user `756aec42473f4bebb15f8d43fe16504d`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:40:51.133162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a3fa0c99-2eb8-41fa-ac1b-769e25c16d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:40:51.158396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:40:51.228977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 652/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Tue Oct 4 12:40:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:40:51.239161 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c7ad8-9c9a-48aa-8e6c-40fb927e7ee3 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:40:51.241055 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-c93c7ad8-9c9a-48aa-8e6c-40fb927e7ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:40:51.363834 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c7ad8-9c9a-48aa-8e6c-40fb927e7ee3 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:40:51.449170 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17dd4a-05b6-4019-907d-b47985821380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:40:51.449696 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17dd4a-05b6-4019-907d-b47985821380 service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:40:51.450024 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17dd4a-05b6-4019-907d-b47985821380 service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:40:51.450316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17dd4a-05b6-4019-907d-b47985821380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:40:51.451943 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-9d17dd4a-05b6-4019-907d-b47985821380 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:40:51.516923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:40:51.579251 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17dd4a-05b6-4019-907d-b47985821380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:40:51.580708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17dd4a-05b6-4019-907d-b47985821380 service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:40:51.601324 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 659/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:40:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:40:54.125817 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68798f40-b326-4ce7-859a-da1b854ae9db None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:40:54.126465 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68798f40-b326-4ce7-859a-da1b854ae9db None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:40:54.126465 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68798f40-b326-4ce7-859a-da1b854ae9db None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:40:54.127777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 653/1312] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:40:54.133538 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde8c31-6c16-45bc-923e-53a257688451 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:40:54.134072 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde8c31-6c16-45bc-923e-53a257688451 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:40:54.134169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde8c31-6c16-45bc-923e-53a257688451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:40:54.215797 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-2bde8c31-6c16-45bc-923e-53a257688451 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:40:54.216949 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-2bde8c31-6c16-45bc-923e-53a257688451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:40:54.327508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 660/1313] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:40:54.609491 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68798f40-b326-4ce7-859a-da1b854ae9db None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:40:54.611072 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-68798f40-b326-4ce7-859a-da1b854ae9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:40:54.730376 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68798f40-b326-4ce7-859a-da1b854ae9db None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:40:54.809552 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:40:54.812380 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93aff4-05b7-4f86-847c-f1f37c4b0533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:40:54.813091 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93aff4-05b7-4f86-847c-f1f37c4b0533 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:40:54.813653 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93aff4-05b7-4f86-847c-f1f37c4b0533 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:40:54.813993 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93aff4-05b7-4f86-847c-f1f37c4b0533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:40:54.816127 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0a93aff4-05b7-4f86-847c-f1f37c4b0533 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:40:54.941904 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93aff4-05b7-4f86-847c-f1f37c4b0533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:40:54.943336 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93aff4-05b7-4f86-847c-f1f37c4b0533 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:40:54.963256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 654/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:40:57.506228 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8149ee-2099-457b-9dd6-c7d80f6afac7 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:40:57.506783 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8149ee-2099-457b-9dd6-c7d80f6afac7 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:40:57.506783 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8149ee-2099-457b-9dd6-c7d80f6afac7 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:40:57.508209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 661/1315] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:40:57.513713 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573a932-79b2-44c6-8d17-255b519ccb4b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:40:57.514164 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573a932-79b2-44c6-8d17-255b519ccb4b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:40:57.514276 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573a932-79b2-44c6-8d17-255b519ccb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:40:57.584238 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f573a932-79b2-44c6-8d17-255b519ccb4b None None] MFA Rules not processed for user `7563b6e0ea29491291eb70a0be54b165`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:40:57.585466 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f573a932-79b2-44c6-8d17-255b519ccb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:40:57.641713 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:40:57.689272 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 655/1316] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:40:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:40:57.976082 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8149ee-2099-457b-9dd6-c7d80f6afac7 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:40:57.977637 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-3a8149ee-2099-457b-9dd6-c7d80f6afac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:40:57.989537 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:40:58.091224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8149ee-2099-457b-9dd6-c7d80f6afac7 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:40:58.176144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c1bad-648f-423e-90ee-b2d3547dc40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:40:58.176662 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c1bad-648f-423e-90ee-b2d3547dc40e service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:40:58.176975 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c1bad-648f-423e-90ee-b2d3547dc40e service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:40:58.177265 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c1bad-648f-423e-90ee-b2d3547dc40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:40:58.178962 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-af5c1bad-648f-423e-90ee-b2d3547dc40e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:40:58.244770 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:40:58.299686 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c1bad-648f-423e-90ee-b2d3547dc40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:40:58.301137 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c1bad-648f-423e-90ee-b2d3547dc40e service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:40:58.319986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 662/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:40:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:41:01.120527 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02ef82-043f-482b-8e8c-d0dc18fa57bc None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:01.121107 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02ef82-043f-482b-8e8c-d0dc18fa57bc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:01.121107 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02ef82-043f-482b-8e8c-d0dc18fa57bc None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:01.122994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 656/1318] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:41:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:01.128420 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dd180-4217-4251-a86f-e12fc4deb0f0 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:01.128854 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dd180-4217-4251-a86f-e12fc4deb0f0 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:01.128969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dd180-4217-4251-a86f-e12fc4deb0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:01.194632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-ea2dd180-4217-4251-a86f-e12fc4deb0f0 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:01.195849 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-ea2dd180-4217-4251-a86f-e12fc4deb0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:01.298439 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 663/1319] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:41:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:01.592847 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e16e93-d244-4279-8264-3a249e718638 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:01.593151 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e16e93-d244-4279-8264-3a249e718638 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:01.593420 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e16e93-d244-4279-8264-3a249e718638 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:01.594782 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 657/1320] 10.222.0.33 () {54 vars in 964 bytes} [Tue Oct 4 12:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:01.600522 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68158c7-020e-4c75-8feb-75012a93f113 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:01.600810 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68158c7-020e-4c75-8feb-75012a93f113 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:01.601084 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68158c7-020e-4c75-8feb-75012a93f113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:01.624869 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:01.679181 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-a68158c7-020e-4c75-8feb-75012a93f113 None None] MFA Rules not processed for user `756aec42473f4bebb15f8d43fe16504d`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:01.680387 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-a68158c7-020e-4c75-8feb-75012a93f113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:01.753911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 664/1321] 10.222.0.33 () {58 vars in 1061 bytes} [Tue Oct 4 12:41:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:01.763222 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e16e93-d244-4279-8264-3a249e718638 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:41:01.764748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-63e16e93-d244-4279-8264-3a249e718638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:01.868073 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e16e93-d244-4279-8264-3a249e718638 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:41:01.945636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb29c0-b2ac-42bc-85ec-866fd2d58f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:41:01.946277 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb29c0-b2ac-42bc-85ec-866fd2d58f77 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:01.946552 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb29c0-b2ac-42bc-85ec-866fd2d58f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:01.946821 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb29c0-b2ac-42bc-85ec-866fd2d58f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:01.948410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e7fb29c0-b2ac-42bc-85ec-866fd2d58f77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:01.955972 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:02.086168 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb29c0-b2ac-42bc-85ec-866fd2d58f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:41:02.087615 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb29c0-b2ac-42bc-85ec-866fd2d58f77 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:41:02.106719 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 658/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:41:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:41:05.411456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f2d9e-6d7b-44e8-8c42-ea54db1f4391 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:05.412606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f2d9e-6d7b-44e8-8c42-ea54db1f4391 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:05.412606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f2d9e-6d7b-44e8-8c42-ea54db1f4391 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:05.414643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 665/1323] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:41:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:05.421776 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090867-74e4-469d-bc35-3edbfd9b956d None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:05.422397 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090867-74e4-469d-bc35-3edbfd9b956d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:05.422543 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090867-74e4-469d-bc35-3edbfd9b956d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:05.507831 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3c090867-74e4-469d-bc35-3edbfd9b956d None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:05.509063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3c090867-74e4-469d-bc35-3edbfd9b956d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:05.577857 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:05.628777 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 659/1324] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:05.923450 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f2d9e-6d7b-44e8-8c42-ea54db1f4391 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:41:05.925009 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-5d8f2d9e-6d7b-44e8-8c42-ea54db1f4391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:06.033251 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f2d9e-6d7b-44e8-8c42-ea54db1f4391 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:41:06.046402 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:06.110519 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379cc95-5b13-4b02-b550-1b7da5e9c9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:41:06.111074 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379cc95-5b13-4b02-b550-1b7da5e9c9da service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:06.111356 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379cc95-5b13-4b02-b550-1b7da5e9c9da service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:06.111631 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379cc95-5b13-4b02-b550-1b7da5e9c9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:06.113245 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-7379cc95-5b13-4b02-b550-1b7da5e9c9da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:06.226449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379cc95-5b13-4b02-b550-1b7da5e9c9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:41:06.227813 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379cc95-5b13-4b02-b550-1b7da5e9c9da service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:41:06.246935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 666/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:41:08.478055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dcf98-ddef-49aa-9f20-1a05ebc913ae None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:08.478630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dcf98-ddef-49aa-9f20-1a05ebc913ae None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:08.478630 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dcf98-ddef-49aa-9f20-1a05ebc913ae None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:08.480057 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 660/1326] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:41:08.486504 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1575df-bb92-4245-88a5-3477f8f355a4 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:08.487021 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1575df-bb92-4245-88a5-3477f8f355a4 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:08.487021 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1575df-bb92-4245-88a5-3477f8f355a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:08.538408 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:08.558445 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-df1575df-bb92-4245-88a5-3477f8f355a4 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:08.560239 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-df1575df-bb92-4245-88a5-3477f8f355a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:08.669236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 667/1327] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:08.950487 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dcf98-ddef-49aa-9f20-1a05ebc913ae None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:41:08.952017 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b61dcf98-ddef-49aa-9f20-1a05ebc913ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:09.058504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dcf98-ddef-49aa-9f20-1a05ebc913ae None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:41:09.134548 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d9090-ef53-4c0e-9349-ce04b28c0a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:41:09.135299 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d9090-ef53-4c0e-9349-ce04b28c0a35 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:09.135579 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d9090-ef53-4c0e-9349-ce04b28c0a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:09.135841 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d9090-ef53-4c0e-9349-ce04b28c0a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:09.137468 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7d1d9090-ef53-4c0e-9349-ce04b28c0a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:09.155502 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:09.261152 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d9090-ef53-4c0e-9349-ce04b28c0a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:41:09.262868 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d9090-ef53-4c0e-9349-ce04b28c0a35 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:41:09.282483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 661/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:41:11.552791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ca7f7-9b4c-412f-8ce2-cc1ddc08b6ed None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:11.553398 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ca7f7-9b4c-412f-8ce2-cc1ddc08b6ed None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:11.553398 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ca7f7-9b4c-412f-8ce2-cc1ddc08b6ed None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:11.554684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 668/1329] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:11.560461 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7353f0-1194-4b90-a233-6f7a88b36f32 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:11.560962 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7353f0-1194-4b90-a233-6f7a88b36f32 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:11.561071 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7353f0-1194-4b90-a233-6f7a88b36f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:11.633652 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9e7353f0-1194-4b90-a233-6f7a88b36f32 None None] MFA Rules not processed for user `7563b6e0ea29491291eb70a0be54b165`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:11.634932 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9e7353f0-1194-4b90-a233-6f7a88b36f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:11.707919 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:11.741883 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 662/1330] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:41:11 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:12.028417 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ca7f7-9b4c-412f-8ce2-cc1ddc08b6ed None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:41:12.030112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-7a5ca7f7-9b4c-412f-8ce2-cc1ddc08b6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:12.158532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ca7f7-9b4c-412f-8ce2-cc1ddc08b6ed None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:41:12.183519 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:12.256461 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc12f2-6d47-47e0-9e73-d617cd729410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:41:12.257018 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc12f2-6d47-47e0-9e73-d617cd729410 service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:12.257569 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc12f2-6d47-47e0-9e73-d617cd729410 service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:12.257957 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc12f2-6d47-47e0-9e73-d617cd729410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:12.259919 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-a9bc12f2-6d47-47e0-9e73-d617cd729410 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:12.397876 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc12f2-6d47-47e0-9e73-d617cd729410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:41:12.400602 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc12f2-6d47-47e0-9e73-d617cd729410 service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:41:12.423208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 669/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:41:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:41:14.909741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a488c63-66f3-412f-811d-1fe8a104ed82 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:14.910311 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a488c63-66f3-412f-811d-1fe8a104ed82 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:14.910311 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a488c63-66f3-412f-811d-1fe8a104ed82 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:14.911665 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 663/1332] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:14.917118 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc34007-8cfd-4aa9-84a7-f7c053c939cf None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:14.917575 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc34007-8cfd-4aa9-84a7-f7c053c939cf None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:14.917703 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc34007-8cfd-4aa9-84a7-f7c053c939cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:14.985361 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:14.988693 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-9dc34007-8cfd-4aa9-84a7-f7c053c939cf None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:14.990316 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-9dc34007-8cfd-4aa9-84a7-f7c053c939cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:15.086224 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 670/1333] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:15.366817 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a488c63-66f3-412f-811d-1fe8a104ed82 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:41:15.368365 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8a488c63-66f3-412f-811d-1fe8a104ed82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:15.475277 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a488c63-66f3-412f-811d-1fe8a104ed82 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:41:15.549451 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee6930-9f29-4dfd-824a-599850274c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:41:15.549972 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee6930-9f29-4dfd-824a-599850274c30 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:15.550275 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee6930-9f29-4dfd-824a-599850274c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:15.550540 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee6930-9f29-4dfd-824a-599850274c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:15.552129 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7fee6930-9f29-4dfd-824a-599850274c30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:15.579902 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:15.690967 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee6930-9f29-4dfd-824a-599850274c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:41:15.693687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee6930-9f29-4dfd-824a-599850274c30 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:41:15.726467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 664/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:41:17.984228 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a687e7-2128-47f6-a82b-99bf0d611a1a None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:17.984825 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a687e7-2128-47f6-a82b-99bf0d611a1a None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:17.984825 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a687e7-2128-47f6-a82b-99bf0d611a1a None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:17.986131 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 671/1335] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:17.992474 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69f455-66f7-4ef4-ba90-dcacf5949c60 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:17.993043 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69f455-66f7-4ef4-ba90-dcacf5949c60 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:17.993167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69f455-66f7-4ef4-ba90-dcacf5949c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:18.072651 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ec69f455-66f7-4ef4-ba90-dcacf5949c60 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:18.074594 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ec69f455-66f7-4ef4-ba90-dcacf5949c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:18.158063 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:18.184489 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 665/1336] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:18.470997 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a687e7-2128-47f6-a82b-99bf0d611a1a None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:41:18.472545 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-e9a687e7-2128-47f6-a82b-99bf0d611a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:18.582384 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a687e7-2128-47f6-a82b-99bf0d611a1a None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:41:18.612349 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:18.659389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a56f00-6d12-4d50-9584-fcd0def2d889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:41:18.659991 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a56f00-6d12-4d50-9584-fcd0def2d889 service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:18.660263 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a56f00-6d12-4d50-9584-fcd0def2d889 service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:18.660528 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a56f00-6d12-4d50-9584-fcd0def2d889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:18.662191 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-e3a56f00-6d12-4d50-9584-fcd0def2d889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:18.773339 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a56f00-6d12-4d50-9584-fcd0def2d889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:41:18.774824 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a56f00-6d12-4d50-9584-fcd0def2d889 service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:41:18.793859 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 672/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:41:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:41:21.353534 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f96b91-52de-48f2-afc8-238c9d3a0e09 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:21.354130 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f96b91-52de-48f2-afc8-238c9d3a0e09 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:21.354130 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f96b91-52de-48f2-afc8-238c9d3a0e09 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:21.356082 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 666/1338] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:41:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:21.361646 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf7fbd-f7a9-46bd-bf90-eccffffba841 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:21.362125 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf7fbd-f7a9-46bd-bf90-eccffffba841 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:21.362216 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf7fbd-f7a9-46bd-bf90-eccffffba841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:21.433962 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-c7cf7fbd-f7a9-46bd-bf90-eccffffba841 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:21.435215 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-c7cf7fbd-f7a9-46bd-bf90-eccffffba841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:21.444184 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:21.538135 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 673/1339] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:41:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:21.818546 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f96b91-52de-48f2-afc8-238c9d3a0e09 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:41:21.820088 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-55f96b91-52de-48f2-afc8-238c9d3a0e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:21.935621 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f96b91-52de-48f2-afc8-238c9d3a0e09 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:41:22.018588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043a8d4-21de-471a-a066-84b8b54dcbda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:41:22.019203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043a8d4-21de-471a-a066-84b8b54dcbda service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:22.019482 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043a8d4-21de-471a-a066-84b8b54dcbda service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:22.019756 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043a8d4-21de-471a-a066-84b8b54dcbda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:22.021408 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1043a8d4-21de-471a-a066-84b8b54dcbda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:22.061482 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:22.139573 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043a8d4-21de-471a-a066-84b8b54dcbda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:41:22.140921 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043a8d4-21de-471a-a066-84b8b54dcbda service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:41:22.162409 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 667/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:41:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:41:26.428198 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a5e48-149e-4f7f-a73c-c81020369e1f None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:26.428741 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a5e48-149e-4f7f-a73c-c81020369e1f None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:26.428741 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a5e48-149e-4f7f-a73c-c81020369e1f None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:26.430138 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 674/1341] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:26.436958 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62485e88-ebcb-46d7-9688-90d442bece22 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:26.437548 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62485e88-ebcb-46d7-9688-90d442bece22 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:26.437988 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62485e88-ebcb-46d7-9688-90d442bece22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:26.530007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-62485e88-ebcb-46d7-9688-90d442bece22 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:26.531278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-62485e88-ebcb-46d7-9688-90d442bece22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:26.645695 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:26.665096 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 668/1342] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:26.943891 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a5e48-149e-4f7f-a73c-c81020369e1f None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:41:26.945688 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-3a7a5e48-149e-4f7f-a73c-c81020369e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:27.135144 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a5e48-149e-4f7f-a73c-c81020369e1f None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:41:27.201033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:27.241759 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1dd8e-8f99-4e85-8808-74510d70618a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:41:27.242715 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1dd8e-8f99-4e85-8808-74510d70618a service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:27.243114 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1dd8e-8f99-4e85-8808-74510d70618a service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:27.243454 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1dd8e-8f99-4e85-8808-74510d70618a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:27.245781 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-f5c1dd8e-8f99-4e85-8808-74510d70618a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:27.405872 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1dd8e-8f99-4e85-8808-74510d70618a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:41:27.407653 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1dd8e-8f99-4e85-8808-74510d70618a service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:41:27.431449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 675/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:41:30.396172 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27356fd2-a697-43c6-9697-f9d7796910e3 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:30.396748 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27356fd2-a697-43c6-9697-f9d7796910e3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:30.397133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27356fd2-a697-43c6-9697-f9d7796910e3 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:30.399750 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 669/1344] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:41:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:30.406780 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e79cc9-4230-4fdf-ba98-bd0812658505 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:30.407351 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e79cc9-4230-4fdf-ba98-bd0812658505 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:30.407724 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e79cc9-4230-4fdf-ba98-bd0812658505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:30.542591 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-85e79cc9-4230-4fdf-ba98-bd0812658505 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:30.544669 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-85e79cc9-4230-4fdf-ba98-bd0812658505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:30.580304 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:30.733295 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 676/1345] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:41:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:31.023379 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27356fd2-a697-43c6-9697-f9d7796910e3 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:41:31.025208 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-27356fd2-a697-43c6-9697-f9d7796910e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:31.177316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27356fd2-a697-43c6-9697-f9d7796910e3 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:41:31.292965 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcd893-c0fc-477d-b0a2-5d1d569d0184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:41:31.293729 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcd893-c0fc-477d-b0a2-5d1d569d0184 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:31.294290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcd893-c0fc-477d-b0a2-5d1d569d0184 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:31.294691 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcd893-c0fc-477d-b0a2-5d1d569d0184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:31.296795 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1afcd893-c0fc-477d-b0a2-5d1d569d0184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:31.356317 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:31.498457 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcd893-c0fc-477d-b0a2-5d1d569d0184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:41:31.499931 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcd893-c0fc-477d-b0a2-5d1d569d0184 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:41:31.522682 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 670/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:41:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:41:36.110579 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec52a8e-1fd2-4264-8ace-18f400f30afc None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:36.111076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec52a8e-1fd2-4264-8ace-18f400f30afc None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:36.111076 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec52a8e-1fd2-4264-8ace-18f400f30afc None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:36.112662 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 677/1347] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:36.118636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d7b80-efbf-45f2-865c-a4db17f12527 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:36.118636 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d7b80-efbf-45f2-865c-a4db17f12527 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:36.119315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d7b80-efbf-45f2-865c-a4db17f12527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:36.199008 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-355d7b80-efbf-45f2-865c-a4db17f12527 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:36.200469 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-355d7b80-efbf-45f2-865c-a4db17f12527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:36.314325 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:36.317491 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 671/1348] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:36.604969 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec52a8e-1fd2-4264-8ace-18f400f30afc None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:41:36.606585 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-4ec52a8e-1fd2-4264-8ace-18f400f30afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:36.714388 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec52a8e-1fd2-4264-8ace-18f400f30afc None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:41:36.774567 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:36.801598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21efce-3d0e-4a37-8999-d3071e4cd0f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:41:36.802254 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21efce-3d0e-4a37-8999-d3071e4cd0f4 service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:36.802532 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21efce-3d0e-4a37-8999-d3071e4cd0f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:36.802803 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21efce-3d0e-4a37-8999-d3071e4cd0f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:36.804457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-ac21efce-3d0e-4a37-8999-d3071e4cd0f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:36.922000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21efce-3d0e-4a37-8999-d3071e4cd0f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:41:36.923401 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21efce-3d0e-4a37-8999-d3071e4cd0f4 service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:41:36.944929 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 678/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:41:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:41:42.394193 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d3c45-2e50-4a63-9bb6-c3191e55b4b1 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:42.397428 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d3c45-2e50-4a63-9bb6-c3191e55b4b1 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:42.398329 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9d3c45-2e50-4a63-9bb6-c3191e55b4b1 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:42.402395 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 672/1350] 10.222.0.33 () {54 vars in 921 bytes} [Tue Oct 4 12:41:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:42.409899 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ab1bd-662b-4438-a232-6c981a031540 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:42.411471 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ab1bd-662b-4438-a232-6c981a031540 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:42.412446 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ab1bd-662b-4438-a232-6c981a031540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:42.588035 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-ec4ab1bd-662b-4438-a232-6c981a031540 None None] MFA Rules not processed for user `3a442a785ebe45709bce78add0e0e93a`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:42.590668 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-ec4ab1bd-662b-4438-a232-6c981a031540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:42.643634 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:42.861570 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 679/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Tue Oct 4 12:41:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:47.453124 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669eadf-62b4-4df4-8a5e-5556de3f8322 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:47.454078 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669eadf-62b4-4df4-8a5e-5556de3f8322 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:47.454536 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669eadf-62b4-4df4-8a5e-5556de3f8322 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:47.456421 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 673/1352] 10.222.0.33 () {54 vars in 921 bytes} [Tue Oct 4 12:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:47.461648 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228cb13-f942-44f4-b67e-03b85111e84e None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:47.462182 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228cb13-f942-44f4-b67e-03b85111e84e None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:47.462414 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228cb13-f942-44f4-b67e-03b85111e84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:47.559389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-1228cb13-f942-44f4-b67e-03b85111e84e None None] MFA Rules not processed for user `3a442a785ebe45709bce78add0e0e93a`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:47.560584 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-1228cb13-f942-44f4-b67e-03b85111e84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:47.650283 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:47.702488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 680/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Tue Oct 4 12:41:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:49.396998 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1b3f0-eac1-4a25-9f9b-ffd59b8fb5dc None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:49.398094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1b3f0-eac1-4a25-9f9b-ffd59b8fb5dc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:49.398761 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1b3f0-eac1-4a25-9f9b-ffd59b8fb5dc None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:49.401460 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 674/1354] 10.222.0.33 () {54 vars in 921 bytes} [Tue Oct 4 12:41:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:41:49.409683 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cdaeb-35f7-4bd7-9565-d8f80337b2db None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:49.410452 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cdaeb-35f7-4bd7-9565-d8f80337b2db None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:49.410452 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cdaeb-35f7-4bd7-9565-d8f80337b2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:49.499809 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-e43cdaeb-35f7-4bd7-9565-d8f80337b2db None None] MFA Rules not processed for user `3a442a785ebe45709bce78add0e0e93a`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:49.501198 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-e43cdaeb-35f7-4bd7-9565-d8f80337b2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:49.660517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 681/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Tue Oct 4 12:41:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:53.239057 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faa1da-65ac-4ae1-99eb-096f1d5f82bf None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:53.239696 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faa1da-65ac-4ae1-99eb-096f1d5f82bf None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:53.239696 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faa1da-65ac-4ae1-99eb-096f1d5f82bf None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:53.240928 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 675/1356] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:41:53.246921 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276327b-683c-4455-9b61-9e90f71870c3 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:53.247750 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276327b-683c-4455-9b61-9e90f71870c3 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:53.248209 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276327b-683c-4455-9b61-9e90f71870c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:53.325873 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:53.396842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-3276327b-683c-4455-9b61-9e90f71870c3 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:53.398643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-3276327b-683c-4455-9b61-9e90f71870c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:53.513303 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 682/1357] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:53.833812 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db8f87-1d92-4a0a-adcd-98080991264e None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:53.834852 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db8f87-1d92-4a0a-adcd-98080991264e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:53.836274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db8f87-1d92-4a0a-adcd-98080991264e None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:53.842877 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 676/1358] 10.222.0.33 () {54 vars in 961 bytes} [Tue Oct 4 12:41:53 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:53.856375 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75106a9d-9373-4530-bcc0-d912c6651293 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:53.857681 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75106a9d-9373-4530-bcc0-d912c6651293 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:53.858604 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75106a9d-9373-4530-bcc0-d912c6651293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:54.011433 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-75106a9d-9373-4530-bcc0-d912c6651293 None None] MFA Rules not processed for user `02260480264b4f8f8a7e5cb647ac5d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:54.013158 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-75106a9d-9373-4530-bcc0-d912c6651293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:54.072156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:54.153782 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 683/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Tue Oct 4 12:41:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:54.162324 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db8f87-1d92-4a0a-adcd-98080991264e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:41:54.164295 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-56db8f87-1d92-4a0a-adcd-98080991264e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:54.348687 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db8f87-1d92-4a0a-adcd-98080991264e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:41:54.438133 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:54.484804 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16e731-8347-4d06-a3cc-b90324215e55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:41:54.485520 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16e731-8347-4d06-a3cc-b90324215e55 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:54.485866 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16e731-8347-4d06-a3cc-b90324215e55 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:54.486246 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16e731-8347-4d06-a3cc-b90324215e55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:54.487953 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6e16e731-8347-4d06-a3cc-b90324215e55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:54.596005 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16e731-8347-4d06-a3cc-b90324215e55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:41:54.597733 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16e731-8347-4d06-a3cc-b90324215e55 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:41:54.618689 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 677/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:41:57.260457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf512489-e32e-4bf1-ae87-9e2cf1146d0d None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:57.260457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf512489-e32e-4bf1-ae87-9e2cf1146d0d None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:57.260457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf512489-e32e-4bf1-ae87-9e2cf1146d0d None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:57.260457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 684/1361] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:41:57.268083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4310ead-f017-4f28-9fc4-687ec645c92f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:57.268083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4310ead-f017-4f28-9fc4-687ec645c92f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:57.268083 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4310ead-f017-4f28-9fc4-687ec645c92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:57.354003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f4310ead-f017-4f28-9fc4-687ec645c92f None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:41:57.357891 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f4310ead-f017-4f28-9fc4-687ec645c92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:57.421108 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:57.502135 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 678/1362] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:41:57.798533 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf512489-e32e-4bf1-ae87-9e2cf1146d0d None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:41:57.800476 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-bf512489-e32e-4bf1-ae87-9e2cf1146d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:58.027286 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf512489-e32e-4bf1-ae87-9e2cf1146d0d None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:41:58.045015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:41:58.177706 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d15d5-1fb2-455a-824a-53b1e6ae0b33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:41:58.178798 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d15d5-1fb2-455a-824a-53b1e6ae0b33 service nova] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:41:58.179281 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d15d5-1fb2-455a-824a-53b1e6ae0b33 service nova] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:41:58.179552 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d15d5-1fb2-455a-824a-53b1e6ae0b33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:41:58.181440 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-aa8d15d5-1fb2-455a-824a-53b1e6ae0b33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:41:58.285051 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d15d5-1fb2-455a-824a-53b1e6ae0b33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:41:58.286765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d15d5-1fb2-455a-824a-53b1e6ae0b33 service nova] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:41:58.308279 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 685/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:00.874342 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e4dd1-55e6-4559-a290-e1788f29b252 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:00.874972 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e4dd1-55e6-4559-a290-e1788f29b252 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:00.874972 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e4dd1-55e6-4559-a290-e1788f29b252 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:00.878080 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 679/1364] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:00.882555 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaef7a8-ce9f-44de-9d59-ba784ae1c821 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:00.883823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaef7a8-ce9f-44de-9d59-ba784ae1c821 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:00.883823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaef7a8-ce9f-44de-9d59-ba784ae1c821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:00.900067 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:00.964935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-5aaef7a8-ce9f-44de-9d59-ba784ae1c821 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:00.966376 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-5aaef7a8-ce9f-44de-9d59-ba784ae1c821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:01.089380 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 686/1365] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:01.394557 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1dec9f-e7fb-4f0c-86f2-ce4a55dca542 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:01.394840 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1dec9f-e7fb-4f0c-86f2-ce4a55dca542 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:01.395114 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1dec9f-e7fb-4f0c-86f2-ce4a55dca542 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:01.396640 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 680/1366] 10.222.0.33 () {54 vars in 961 bytes} [Tue Oct 4 12:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:01.405359 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bcb6b-bd22-4748-b149-18823eff9c0a None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:01.405764 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bcb6b-bd22-4748-b149-18823eff9c0a None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:01.406244 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bcb6b-bd22-4748-b149-18823eff9c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:01.499000 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-1b5bcb6b-bd22-4748-b149-18823eff9c0a None None] MFA Rules not processed for user `02260480264b4f8f8a7e5cb647ac5d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:01.500292 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-1b5bcb6b-bd22-4748-b149-18823eff9c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:01.544216 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:01.666935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 687/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Tue Oct 4 12:42:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:01.676454 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1dec9f-e7fb-4f0c-86f2-ce4a55dca542 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:01.678349 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4a1dec9f-e7fb-4f0c-86f2-ce4a55dca542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:01.860645 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1dec9f-e7fb-4f0c-86f2-ce4a55dca542 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:01.883462 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:01.987406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03254dfa-cc5c-458c-9ba8-2aa2af4da20b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:01.988070 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03254dfa-cc5c-458c-9ba8-2aa2af4da20b service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:01.988403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03254dfa-cc5c-458c-9ba8-2aa2af4da20b service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:01.988628 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03254dfa-cc5c-458c-9ba8-2aa2af4da20b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:01.990291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-03254dfa-cc5c-458c-9ba8-2aa2af4da20b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:02.101105 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03254dfa-cc5c-458c-9ba8-2aa2af4da20b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:02.102775 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03254dfa-cc5c-458c-9ba8-2aa2af4da20b service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:02.123796 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 681/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:42:03.018311 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b5c1a-0b89-49f8-afda-3f89d4effd17 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:03.019005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b5c1a-0b89-49f8-afda-3f89d4effd17 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:03.019005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b5c1a-0b89-49f8-afda-3f89d4effd17 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:03.020205 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 688/1369] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:03.027954 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7954b1-b9a2-4c26-bf21-f83ed1f6d6bc None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:03.028502 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7954b1-b9a2-4c26-bf21-f83ed1f6d6bc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:03.028705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7954b1-b9a2-4c26-bf21-f83ed1f6d6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:03.076166 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:03.131112 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-be7954b1-b9a2-4c26-bf21-f83ed1f6d6bc None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:03.132870 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-be7954b1-b9a2-4c26-bf21-f83ed1f6d6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:03.290284 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 682/1370] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:03.745842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867972a-eb0f-4447-ab02-9a64abe393e5 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:03.745842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867972a-eb0f-4447-ab02-9a64abe393e5 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:03.745842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867972a-eb0f-4447-ab02-9a64abe393e5 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:03.745842 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 689/1371] 10.222.0.33 () {54 vars in 963 bytes} [Tue Oct 4 12:42:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:42:03.748562 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b729539-79e5-44b3-a804-41c0bf8b1c3e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:03.748562 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b729539-79e5-44b3-a804-41c0bf8b1c3e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:03.748562 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b729539-79e5-44b3-a804-41c0bf8b1c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:03.877504 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1b729539-79e5-44b3-a804-41c0bf8b1c3e None None] MFA Rules not processed for user `b131edeaa17e4e10af385339d463a512`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:03.878722 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1b729539-79e5-44b3-a804-41c0bf8b1c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:03.944676 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:03.997946 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 683/1372] 10.222.0.33 () {58 vars in 1060 bytes} [Tue Oct 4 12:42:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:04.005553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867972a-eb0f-4447-ab02-9a64abe393e5 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:04.007132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-6867972a-eb0f-4447-ab02-9a64abe393e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:04.151265 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867972a-eb0f-4447-ab02-9a64abe393e5 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:04.206718 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:04.256460 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331b0aa-69ec-4992-a2fb-cc9bd2cdd233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b131edeaa17e4e10af385339d463a512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:04.257402 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331b0aa-69ec-4992-a2fb-cc9bd2cdd233 service cinder] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:04.257914 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331b0aa-69ec-4992-a2fb-cc9bd2cdd233 service cinder] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:04.258437 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331b0aa-69ec-4992-a2fb-cc9bd2cdd233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:04.262335 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-e331b0aa-69ec-4992-a2fb-cc9bd2cdd233 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:04.433710 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331b0aa-69ec-4992-a2fb-cc9bd2cdd233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:04.435606 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331b0aa-69ec-4992-a2fb-cc9bd2cdd233 service cinder] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:04.453609 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d020916-8283-4001-a8a8-1574d111476c None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:04.454181 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d020916-8283-4001-a8a8-1574d111476c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:04.454276 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d020916-8283-4001-a8a8-1574d111476c None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:04.456008 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 684/1373] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:42:04.463626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4cba2-0580-409d-8173-c676848cbf93 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:04.463921 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4cba2-0580-409d-8173-c676848cbf93 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:04.464195 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4cba2-0580-409d-8173-c676848cbf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:04.467094 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 690/1374] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:04.552841 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-0fb4cba2-0580-409d-8173-c676848cbf93 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:04.554202 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0fb4cba2-0580-409d-8173-c676848cbf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:04.690944 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 685/1375] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:05.012477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331b0aa-69ec-4992-a2fb-cc9bd2cdd233 service cinder] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:05.013695 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-e331b0aa-69ec-4992-a2fb-cc9bd2cdd233 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:05.091140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:05.128195 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331b0aa-69ec-4992-a2fb-cc9bd2cdd233 service cinder] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:05.269878 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16880a-6814-4ae1-9470-a9c5b8031439 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:05.270761 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16880a-6814-4ae1-9470-a9c5b8031439 service nova] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:05.271049 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16880a-6814-4ae1-9470-a9c5b8031439 service nova] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:05.271321 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16880a-6814-4ae1-9470-a9c5b8031439 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:05.272999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-dc16880a-6814-4ae1-9470-a9c5b8031439 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:05.372267 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:05.387116 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16880a-6814-4ae1-9470-a9c5b8031439 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:05.388787 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16880a-6814-4ae1-9470-a9c5b8031439 service nova] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:05.413492 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 691/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:07.664038 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a863e-1a48-4d4d-9048-1ccedb8e3eb9 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:07.664631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a863e-1a48-4d4d-9048-1ccedb8e3eb9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:07.664631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a863e-1a48-4d4d-9048-1ccedb8e3eb9 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:07.666574 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 686/1377] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:07.672313 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10416f93-276a-4043-a402-402b1f67b1b7 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:07.672811 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10416f93-276a-4043-a402-402b1f67b1b7 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:07.672922 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10416f93-276a-4043-a402-402b1f67b1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:07.764687 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-10416f93-276a-4043-a402-402b1f67b1b7 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:07.765911 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-10416f93-276a-4043-a402-402b1f67b1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:07.878282 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 692/1378] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:42:08.187987 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a863e-1a48-4d4d-9048-1ccedb8e3eb9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:08.189437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-365a863e-1a48-4d4d-9048-1ccedb8e3eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:08.304661 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:08.456713 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a863e-1a48-4d4d-9048-1ccedb8e3eb9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:08.606682 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e0ec3-333f-4f4c-936d-2e2dfac82a9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:08.607280 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e0ec3-333f-4f4c-936d-2e2dfac82a9a service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:08.607624 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e0ec3-333f-4f4c-936d-2e2dfac82a9a service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:08.607969 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e0ec3-333f-4f4c-936d-2e2dfac82a9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:08.609941 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-620e0ec3-333f-4f4c-936d-2e2dfac82a9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:08.626261 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:08.727551 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e0ec3-333f-4f4c-936d-2e2dfac82a9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:08.730242 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e0ec3-333f-4f4c-936d-2e2dfac82a9a service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:08.754647 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 687/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:42:11.170123 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1071c0-0897-48a3-9e9b-8787e865b48d None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:11.171008 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1071c0-0897-48a3-9e9b-8787e865b48d None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:11.171008 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1071c0-0897-48a3-9e9b-8787e865b48d None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:11.172361 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 693/1380] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:11.177977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b63db0-9b95-4f48-886c-12f86a225423 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:11.178573 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b63db0-9b95-4f48-886c-12f86a225423 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:11.178573 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b63db0-9b95-4f48-886c-12f86a225423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:11.272513 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e2b63db0-9b95-4f48-886c-12f86a225423 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:11.274169 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e2b63db0-9b95-4f48-886c-12f86a225423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:11.423818 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:11.445553 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 688/1381] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:11.762123 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1071c0-0897-48a3-9e9b-8787e865b48d None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:11.763852 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-eb1071c0-0897-48a3-9e9b-8787e865b48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:11.840445 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:11.980754 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1071c0-0897-48a3-9e9b-8787e865b48d None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:12.214554 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494e654-4c69-4d68-bf1c-8d45686e549c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:12.216330 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494e654-4c69-4d68-bf1c-8d45686e549c service nova] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:12.216739 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494e654-4c69-4d68-bf1c-8d45686e549c service nova] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:12.216875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494e654-4c69-4d68-bf1c-8d45686e549c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:12.218988 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-5494e654-4c69-4d68-bf1c-8d45686e549c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:12.260694 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:12.425337 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494e654-4c69-4d68-bf1c-8d45686e549c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:12.429190 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494e654-4c69-4d68-bf1c-8d45686e549c service nova] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:12.474562 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 694/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:15.466643 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5b29f-a1ff-4feb-a2fb-21299470f41d None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:15.467350 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5b29f-a1ff-4feb-a2fb-21299470f41d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:15.467350 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5b29f-a1ff-4feb-a2fb-21299470f41d None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:15.468477 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 689/1383] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:15.474375 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c726ab-f1c9-4d0d-ab53-04634e2c2b5f None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:15.475012 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c726ab-f1c9-4d0d-ab53-04634e2c2b5f None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:15.475012 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c726ab-f1c9-4d0d-ab53-04634e2c2b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:15.568208 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-32c726ab-f1c9-4d0d-ab53-04634e2c2b5f None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:15.569617 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-32c726ab-f1c9-4d0d-ab53-04634e2c2b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:15.706758 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:15.714492 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 695/1384] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:16.005585 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5b29f-a1ff-4feb-a2fb-21299470f41d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:16.007163 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6ef5b29f-a1ff-4feb-a2fb-21299470f41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:16.235790 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5b29f-a1ff-4feb-a2fb-21299470f41d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:16.327292 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:16.432421 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251434f-c0b6-4b2d-9722-ab5c21a397a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:16.433039 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251434f-c0b6-4b2d-9722-ab5c21a397a8 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:16.433309 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251434f-c0b6-4b2d-9722-ab5c21a397a8 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:16.433587 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251434f-c0b6-4b2d-9722-ab5c21a397a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:16.435266 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5251434f-c0b6-4b2d-9722-ab5c21a397a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:16.565940 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251434f-c0b6-4b2d-9722-ab5c21a397a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:16.567623 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251434f-c0b6-4b2d-9722-ab5c21a397a8 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:16.588971 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 690/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:18.879159 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e06c3-a45e-4a68-a161-3e8d0de2ceae None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:18.879765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e06c3-a45e-4a68-a161-3e8d0de2ceae None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:18.880236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710e06c3-a45e-4a68-a161-3e8d0de2ceae None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:18.883005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 696/1386] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:42:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:18.889054 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f95937-cc0a-458a-8ae0-b156a3e7a8f3 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:18.889904 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f95937-cc0a-458a-8ae0-b156a3e7a8f3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:18.889904 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f95937-cc0a-458a-8ae0-b156a3e7a8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:18.969662 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e9f95937-cc0a-458a-8ae0-b156a3e7a8f3 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:18.970928 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e9f95937-cc0a-458a-8ae0-b156a3e7a8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:19.009277 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:19.135371 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 691/1387] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:42:19.625873 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e06c3-a45e-4a68-a161-3e8d0de2ceae None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:19.627618 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-710e06c3-a45e-4a68-a161-3e8d0de2ceae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:19.844030 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710e06c3-a45e-4a68-a161-3e8d0de2ceae None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:19.933704 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:19.996475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6828168-10d9-42e6-b83f-d216c427e28f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:19.997086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6828168-10d9-42e6-b83f-d216c427e28f service nova] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:19.997359 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6828168-10d9-42e6-b83f-d216c427e28f service nova] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:19.997630 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6828168-10d9-42e6-b83f-d216c427e28f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:19.999438 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-c6828168-10d9-42e6-b83f-d216c427e28f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:20.110603 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6828168-10d9-42e6-b83f-d216c427e28f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:20.112285 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6828168-10d9-42e6-b83f-d216c427e28f service nova] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:20.135624 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 697/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:20.234280 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae107986-e6dc-4e30-848e-3daa35555e39 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:20.234897 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae107986-e6dc-4e30-848e-3daa35555e39 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:20.234897 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae107986-e6dc-4e30-848e-3daa35555e39 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:20.236249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 692/1389] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:20.242458 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb378f-3919-476f-a6da-aeb9f4a954a5 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:20.242907 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb378f-3919-476f-a6da-aeb9f4a954a5 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:20.243270 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb378f-3919-476f-a6da-aeb9f4a954a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:20.328072 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-02bb378f-3919-476f-a6da-aeb9f4a954a5 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:20.329345 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-02bb378f-3919-476f-a6da-aeb9f4a954a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:20.377805 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:20.452044 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 698/1390] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:20.764916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62082eb8-57b3-4d79-aed6-c652c7951075 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:20.765435 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62082eb8-57b3-4d79-aed6-c652c7951075 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:20.765920 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62082eb8-57b3-4d79-aed6-c652c7951075 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:20.768583 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 693/1391] 10.222.0.33 () {54 vars in 962 bytes} [Tue Oct 4 12:42:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:20.775458 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0735341-e6c3-4642-9281-7c93d2b7bffe None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:20.775951 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0735341-e6c3-4642-9281-7c93d2b7bffe None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:20.776428 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0735341-e6c3-4642-9281-7c93d2b7bffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:20.871982 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-e0735341-e6c3-4642-9281-7c93d2b7bffe None None] MFA Rules not processed for user `f2f4f9541f89433cad52e826794b049f`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:20.873587 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-e0735341-e6c3-4642-9281-7c93d2b7bffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:20.974261 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 699/1392] 10.222.0.33 () {58 vars in 1059 bytes} [Tue Oct 4 12:42:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:42:20.988863 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62082eb8-57b3-4d79-aed6-c652c7951075 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:20.991771 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-62082eb8-57b3-4d79-aed6-c652c7951075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:21.167449 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62082eb8-57b3-4d79-aed6-c652c7951075 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:21.241926 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:21.278855 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e98bfe-178c-468b-a756-2ebb52291375 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f4f9541f89433cad52e826794b049f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:21.280123 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e98bfe-178c-468b-a756-2ebb52291375 service glance] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:21.280642 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e98bfe-178c-468b-a756-2ebb52291375 service glance] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:21.281161 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e98bfe-178c-468b-a756-2ebb52291375 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:21.284455 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c5e98bfe-178c-468b-a756-2ebb52291375 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:21.436254 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e98bfe-178c-468b-a756-2ebb52291375 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:21.437692 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e98bfe-178c-468b-a756-2ebb52291375 service glance] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:21.456484 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 694/1393] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:22.219785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce1124-7e06-43df-b059-53d4e243f7b1 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:22.220295 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce1124-7e06-43df-b059-53d4e243f7b1 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:22.220456 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce1124-7e06-43df-b059-53d4e243f7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:22.271967 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:22.309157 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-86ce1124-7e06-43df-b059-53d4e243f7b1 None None] MFA Rules not processed for user `58d82737c2864faf8fef3b53167c4dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:22.310427 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-86ce1124-7e06-43df-b059-53d4e243f7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:22.421024 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 700/1394] 10.222.0.33 () {58 vars in 1009 bytes} [Tue Oct 4 12:42:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:42:22.468306 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37afec8-b84b-4c77-b1a3-b919a5656760 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:22.468866 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37afec8-b84b-4c77-b1a3-b919a5656760 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:22.468866 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37afec8-b84b-4c77-b1a3-b919a5656760 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:22.470261 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 695/1395] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:22.473801 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc630ae-587f-4159-b77e-d055852b742c None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:22.474173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc630ae-587f-4159-b77e-d055852b742c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:22.474924 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc630ae-587f-4159-b77e-d055852b742c None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:22.475759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d7ab5-96cd-4fbb-afe2-aabf015a4a6f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:22.476046 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d7ab5-96cd-4fbb-afe2-aabf015a4a6f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:22.476318 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d7ab5-96cd-4fbb-afe2-aabf015a4a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:22.476585 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 701/1396] 10.222.0.33 () {54 vars in 943 bytes} [Tue Oct 4 12:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:22.484629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9340ec5-314c-4d70-9667-fdf394efd204 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:22.484917 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9340ec5-314c-4d70-9667-fdf394efd204 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:22.485248 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9340ec5-314c-4d70-9667-fdf394efd204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:22.553232 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6d8d7ab5-96cd-4fbb-afe2-aabf015a4a6f None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:22.554522 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6d8d7ab5-96cd-4fbb-afe2-aabf015a4a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:22.571550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-f9340ec5-314c-4d70-9667-fdf394efd204 None None] MFA Rules not processed for user `332fdff1d02a4664923ddbe0478eaa7a`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:22.573353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-f9340ec5-314c-4d70-9667-fdf394efd204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:22.595167 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:22.644728 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:22.667400 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 696/1397] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:42:22.702886 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 702/1398] 10.222.0.33 () {58 vars in 1040 bytes} [Tue Oct 4 12:42:22 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:42:22.712403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d7ab5-96cd-4fbb-afe2-aabf015a4a6f None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:22.714193 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6d8d7ab5-96cd-4fbb-afe2-aabf015a4a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:22.843290 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d7ab5-96cd-4fbb-afe2-aabf015a4a6f None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:22.928126 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:22.965415 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9340ec5-314c-4d70-9667-fdf394efd204 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:22.966569 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-f9340ec5-314c-4d70-9667-fdf394efd204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:22.988353 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb0561-212e-4554-b873-c762106d9c67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fdff1d02a4664923ddbe0478eaa7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:22.989125 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb0561-212e-4554-b873-c762106d9c67 service swift] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:22.989535 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb0561-212e-4554-b873-c762106d9c67 service swift] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:22.990006 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb0561-212e-4554-b873-c762106d9c67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:22.992360 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-afbb0561-212e-4554-b873-c762106d9c67 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:23.077362 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9340ec5-314c-4d70-9667-fdf394efd204 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:23.103893 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb0561-212e-4554-b873-c762106d9c67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:23.105945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbb0561-212e-4554-b873-c762106d9c67 service swift] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:23.117499 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 697/1399] 10.222.0.33 () {60 vars in 1384 bytes} [Tue Oct 4 12:42:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 12:42:23.159802 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:23.213549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf773b7d-f110-4398-97b2-63ab56857fd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:23.214192 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf773b7d-f110-4398-97b2-63ab56857fd8 service nova] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:23.214466 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf773b7d-f110-4398-97b2-63ab56857fd8 service nova] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:23.214730 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf773b7d-f110-4398-97b2-63ab56857fd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:23.216386 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-cf773b7d-f110-4398-97b2-63ab56857fd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:23.403079 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf773b7d-f110-4398-97b2-63ab56857fd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:23.405597 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf773b7d-f110-4398-97b2-63ab56857fd8 service nova] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:23.431756 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 703/1400] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 12:42:25.842403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35de5d-3dc0-461a-9f2e-07e0485898b7 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:25.843009 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35de5d-3dc0-461a-9f2e-07e0485898b7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:25.843009 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35de5d-3dc0-461a-9f2e-07e0485898b7 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:25.847427 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 698/1401] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:25.853142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abf567-a2bd-4797-9b53-35a4491d33f5 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:25.853142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abf567-a2bd-4797-9b53-35a4491d33f5 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:25.853142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abf567-a2bd-4797-9b53-35a4491d33f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:25.944321 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-42abf567-a2bd-4797-9b53-35a4491d33f5 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:25.945878 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-42abf567-a2bd-4797-9b53-35a4491d33f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:26.001114 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:26.082158 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 704/1402] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:26.372848 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35de5d-3dc0-461a-9f2e-07e0485898b7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:26.374421 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8b35de5d-3dc0-461a-9f2e-07e0485898b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:26.505792 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:26.570234 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35de5d-3dc0-461a-9f2e-07e0485898b7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:26.703508 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d57b3d-50d6-429c-a296-5672ba25882a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:26.704121 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d57b3d-50d6-429c-a296-5672ba25882a service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:26.704396 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d57b3d-50d6-429c-a296-5672ba25882a service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:26.704663 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d57b3d-50d6-429c-a296-5672ba25882a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:26.706328 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-27d57b3d-50d6-429c-a296-5672ba25882a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:26.775114 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:26.811269 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d57b3d-50d6-429c-a296-5672ba25882a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:26.813113 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d57b3d-50d6-429c-a296-5672ba25882a service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:26.834177 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 699/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:29.111230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fee1ea-d956-4e57-aaa6-27ad7b59f9fa None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:29.111826 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fee1ea-d956-4e57-aaa6-27ad7b59f9fa None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:29.111826 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fee1ea-d956-4e57-aaa6-27ad7b59f9fa None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:29.113501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 705/1404] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:42:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:29.121198 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bb303-d1a7-49a4-9a19-39655dfba598 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:29.121752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bb303-d1a7-49a4-9a19-39655dfba598 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:29.121972 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bb303-d1a7-49a4-9a19-39655dfba598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:29.232377 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a43bb303-d1a7-49a4-9a19-39655dfba598 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:29.234005 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a43bb303-d1a7-49a4-9a19-39655dfba598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:29.370573 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 700/1405] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:29.660373 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fee1ea-d956-4e57-aaa6-27ad7b59f9fa None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:29.661938 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-06fee1ea-d956-4e57-aaa6-27ad7b59f9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:29.883699 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fee1ea-d956-4e57-aaa6-27ad7b59f9fa None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:29.920182 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:30.063378 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6df6e-c3ec-4b33-b0d4-4755db51aea0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:30.063378 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6df6e-c3ec-4b33-b0d4-4755db51aea0 service nova] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:30.063378 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6df6e-c3ec-4b33-b0d4-4755db51aea0 service nova] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:30.063378 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6df6e-c3ec-4b33-b0d4-4755db51aea0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:30.067807 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-49d6df6e-c3ec-4b33-b0d4-4755db51aea0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:30.188893 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6df6e-c3ec-4b33-b0d4-4755db51aea0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:30.190773 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6df6e-c3ec-4b33-b0d4-4755db51aea0 service nova] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:30.218538 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 706/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:32.533758 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d8ffe-6c8f-47a0-9dc2-6e6b4044f001 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:32.534371 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d8ffe-6c8f-47a0-9dc2-6e6b4044f001 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:32.534371 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d8ffe-6c8f-47a0-9dc2-6e6b4044f001 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:32.538401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 701/1407] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:42:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:32.541915 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c2cd3-212b-4a97-90ba-e5f761e3a5ae None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:32.542426 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c2cd3-212b-4a97-90ba-e5f761e3a5ae None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:32.542521 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c2cd3-212b-4a97-90ba-e5f761e3a5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:32.583839 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:32.625230 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-598c2cd3-212b-4a97-90ba-e5f761e3a5ae None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:32.626499 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-598c2cd3-212b-4a97-90ba-e5f761e3a5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:32.746328 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 707/1408] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:33.045868 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d8ffe-6c8f-47a0-9dc2-6e6b4044f001 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:33.047420 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f82d8ffe-6c8f-47a0-9dc2-6e6b4044f001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:33.091745 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:33.227934 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d8ffe-6c8f-47a0-9dc2-6e6b4044f001 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:33.378582 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:33.385266 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d044b1-aea2-41fa-a6f4-488089d064a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:33.385886 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d044b1-aea2-41fa-a6f4-488089d064a3 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:33.386196 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d044b1-aea2-41fa-a6f4-488089d064a3 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:33.386467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d044b1-aea2-41fa-a6f4-488089d064a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:33.388261 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-07d044b1-aea2-41fa-a6f4-488089d064a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:33.500437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d044b1-aea2-41fa-a6f4-488089d064a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:33.502060 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d044b1-aea2-41fa-a6f4-488089d064a3 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:33.523683 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 702/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:42.744792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=53 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:43.521134 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=47 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:46.029793 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7f036-ccc7-4d60-9581-86d8f7b5a380 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:46.030389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7f036-ccc7-4d60-9581-86d8f7b5a380 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:46.030389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7f036-ccc7-4d60-9581-86d8f7b5a380 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:46.031690 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 708/1410] 10.222.0.33 () {54 vars in 918 bytes} [Tue Oct 4 12:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:46.037084 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd60e6-c765-4512-9dc6-2ab4738ec171 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:46.037545 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd60e6-c765-4512-9dc6-2ab4738ec171 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:46.037650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd60e6-c765-4512-9dc6-2ab4738ec171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:46.115026 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3bdd60e6-c765-4512-9dc6-2ab4738ec171 None None] MFA Rules not processed for user `3a442a785ebe45709bce78add0e0e93a`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:46.116256 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3bdd60e6-c765-4512-9dc6-2ab4738ec171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:46.224041 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 703/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Tue Oct 4 12:42:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:46.238270 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12731a-11c6-49fa-8671-d83115848cb5 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:46.238551 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12731a-11c6-49fa-8671-d83115848cb5 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:46.238822 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12731a-11c6-49fa-8671-d83115848cb5 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:46.240158 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 709/1412] 10.222.0.33 () {54 vars in 918 bytes} [Tue Oct 4 12:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:46.245211 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f094a-6f8e-4046-85b7-db2ba4c579b0 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:46.245495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f094a-6f8e-4046-85b7-db2ba4c579b0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:46.245771 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f094a-6f8e-4046-85b7-db2ba4c579b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:46.320335 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:46.323626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-db3f094a-6f8e-4046-85b7-db2ba4c579b0 None None] MFA Rules not processed for user `02260480264b4f8f8a7e5cb647ac5d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:46.325404 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-db3f094a-6f8e-4046-85b7-db2ba4c579b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:46.445406 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 704/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Tue Oct 4 12:42:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:49.059663 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224d6f2-a40f-4ca5-878c-6be07db57295 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:49.060235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224d6f2-a40f-4ca5-878c-6be07db57295 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:49.060235 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224d6f2-a40f-4ca5-878c-6be07db57295 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:49.061523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:49.066917 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc26d00-2822-462d-b820-85372060abad None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:49.067355 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc26d00-2822-462d-b820-85372060abad None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:49.067455 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc26d00-2822-462d-b820-85372060abad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:49.142759 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4cc26d00-2822-462d-b820-85372060abad None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:49.144000 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4cc26d00-2822-462d-b820-85372060abad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:49.203515 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:49.248028 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:49.547925 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224d6f2-a40f-4ca5-878c-6be07db57295 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:49.549448 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-f224d6f2-a40f-4ca5-878c-6be07db57295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:49.711331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224d6f2-a40f-4ca5-878c-6be07db57295 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:49.786300 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:49.827274 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453b8c4-a44c-4cf2-a585-045034620142 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:49.827881 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453b8c4-a44c-4cf2-a585-045034620142 service nova] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:49.828173 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453b8c4-a44c-4cf2-a585-045034620142 service nova] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:49.828422 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453b8c4-a44c-4cf2-a585-045034620142 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:49.830088 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-e453b8c4-a44c-4cf2-a585-045034620142 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:49.920085 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453b8c4-a44c-4cf2-a585-045034620142 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:49.921710 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453b8c4-a44c-4cf2-a585-045034620142 service nova] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:49.940115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 711/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:51.111025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cac0e8-6670-4fd1-b2c4-8fe246c91249 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:51.111576 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cac0e8-6670-4fd1-b2c4-8fe246c91249 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:51.111697 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cac0e8-6670-4fd1-b2c4-8fe246c91249 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:51.112953 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 12:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:51.120049 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247e1fc-fdd1-45ed-b700-f069352a1950 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:51.120632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247e1fc-fdd1-45ed-b700-f069352a1950 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:51.120632 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247e1fc-fdd1-45ed-b700-f069352a1950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:51.201518 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-f247e1fc-fdd1-45ed-b700-f069352a1950 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:51.202785 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-f247e1fc-fdd1-45ed-b700-f069352a1950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:51.230003 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:51.308699 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 12:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:51.460250 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cac0e8-6670-4fd1-b2c4-8fe246c91249 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:51.461723 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f1cac0e8-6670-4fd1-b2c4-8fe246c91249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:51.614518 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cac0e8-6670-4fd1-b2c4-8fe246c91249 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:51.656118 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:51.727726 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3706669-54fd-4251-ab89-bcb3b3e504b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:51.728873 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3706669-54fd-4251-ab89-bcb3b3e504b6 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:51.729145 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3706669-54fd-4251-ab89-bcb3b3e504b6 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:51.729411 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3706669-54fd-4251-ab89-bcb3b3e504b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:51.731025 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a3706669-54fd-4251-ab89-bcb3b3e504b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:51.819029 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3706669-54fd-4251-ab89-bcb3b3e504b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:51.820609 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3706669-54fd-4251-ab89-bcb3b3e504b6 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:51.838789 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 707/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:52.982880 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d199837-504d-4d0b-b642-0f00f0a5413e None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:52.983433 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d199837-504d-4d0b-b642-0f00f0a5413e None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:52.983433 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d199837-504d-4d0b-b642-0f00f0a5413e None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:52.984762 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 12:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:52.991364 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd13fe4-2440-4de6-a8f5-54f5c236405c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:52.991907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd13fe4-2440-4de6-a8f5-54f5c236405c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:52.991907 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd13fe4-2440-4de6-a8f5-54f5c236405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:53.049098 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:53.072469 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-abd13fe4-2440-4de6-a8f5-54f5c236405c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:53.073680 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-abd13fe4-2440-4de6-a8f5-54f5c236405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:53.175602 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 12:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:53.329928 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d199837-504d-4d0b-b642-0f00f0a5413e None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:53.331477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-4d199837-504d-4d0b-b642-0f00f0a5413e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:53.595396 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d199837-504d-4d0b-b642-0f00f0a5413e None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:53.608192 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:53.712042 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e62fa-011b-497d-82b6-d4db15abf886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:53.713191 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e62fa-011b-497d-82b6-d4db15abf886 service nova] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:53.713464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e62fa-011b-497d-82b6-d4db15abf886 service nova] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:53.713726 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e62fa-011b-497d-82b6-d4db15abf886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:53.715349 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-8e2e62fa-011b-497d-82b6-d4db15abf886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:53.805171 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e62fa-011b-497d-82b6-d4db15abf886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:53.807389 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e62fa-011b-497d-82b6-d4db15abf886 service nova] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:53.824962 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 714/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:54.944634 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d97ed3-30e4-483e-8708-1531efffdc7e None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:54.945188 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d97ed3-30e4-483e-8708-1531efffdc7e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:54.945188 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d97ed3-30e4-483e-8708-1531efffdc7e None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:54.946529 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 12:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:54.953604 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669def8-6746-4037-8c8c-3061090451f8 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:54.954069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669def8-6746-4037-8c8c-3061090451f8 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:54.954182 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669def8-6746-4037-8c8c-3061090451f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:54.973747 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:55.031048 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-e669def8-6746-4037-8c8c-3061090451f8 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:55.032271 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-e669def8-6746-4037-8c8c-3061090451f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:55.130882 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 12:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:55.282771 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d97ed3-30e4-483e-8708-1531efffdc7e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:55.284529 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d3d97ed3-30e4-483e-8708-1531efffdc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:55.401985 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:55.440388 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d97ed3-30e4-483e-8708-1531efffdc7e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:55.556738 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ee409-3739-433c-a4e8-2b05d3207876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:55.557260 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ee409-3739-433c-a4e8-2b05d3207876 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:55.557535 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ee409-3739-433c-a4e8-2b05d3207876 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:55.557807 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ee409-3739-433c-a4e8-2b05d3207876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:55.559432 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c72ee409-3739-433c-a4e8-2b05d3207876 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:55.635471 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:55.649495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ee409-3739-433c-a4e8-2b05d3207876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:55.651650 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ee409-3739-433c-a4e8-2b05d3207876 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:55.670336 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 710/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:56.798692 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c3f01-6e6d-4106-a50c-ae0bd4005921 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:56.799317 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c3f01-6e6d-4106-a50c-ae0bd4005921 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:56.799317 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c3f01-6e6d-4106-a50c-ae0bd4005921 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:56.800598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 12:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:56.807780 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145f1dd-4005-4960-802b-3bfbddca08f5 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:56.808394 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145f1dd-4005-4960-802b-3bfbddca08f5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:56.808394 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145f1dd-4005-4960-802b-3bfbddca08f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:56.884555 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8145f1dd-4005-4960-802b-3bfbddca08f5 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:56.886027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8145f1dd-4005-4960-802b-3bfbddca08f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:56.983665 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 12:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:57.141426 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c3f01-6e6d-4106-a50c-ae0bd4005921 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:57.142931 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-c23c3f01-6e6d-4106-a50c-ae0bd4005921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:57.232574 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:57.295553 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c3f01-6e6d-4106-a50c-ae0bd4005921 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:57.406207 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e97e37-55d7-4a70-aeed-ee14810dccdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:57.406737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e97e37-55d7-4a70-aeed-ee14810dccdf service nova] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:57.407005 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e97e37-55d7-4a70-aeed-ee14810dccdf service nova] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:57.407266 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e97e37-55d7-4a70-aeed-ee14810dccdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:57.408878 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-99e97e37-55d7-4a70-aeed-ee14810dccdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:57.459336 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:57.501069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e97e37-55d7-4a70-aeed-ee14810dccdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:57.502723 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e97e37-55d7-4a70-aeed-ee14810dccdf service nova] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:57.520985 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 717/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:42:58.584835 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0513c-2267-4fd9-8b27-55546932e0cb None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:58.585419 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0513c-2267-4fd9-8b27-55546932e0cb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:58.585419 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0513c-2267-4fd9-8b27-55546932e0cb None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:58.586774 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 12:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:42:58.593840 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df4394-0d1a-4606-b94e-dbe9b35a03ba None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:58.594352 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df4394-0d1a-4606-b94e-dbe9b35a03ba None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:58.594457 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df4394-0d1a-4606-b94e-dbe9b35a03ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:58.672999 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-67df4394-0d1a-4606-b94e-dbe9b35a03ba None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:42:58.674154 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-67df4394-0d1a-4606-b94e-dbe9b35a03ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:58.775635 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 12:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:42:58.925037 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0513c-2267-4fd9-8b27-55546932e0cb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:42:58.926581 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d4e0513c-2267-4fd9-8b27-55546932e0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:58.977790 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:59.079136 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0513c-2267-4fd9-8b27-55546932e0cb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:42:59.204946 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eee4a7-6abc-4790-9fdd-32a128d85b1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:42:59.204946 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eee4a7-6abc-4790-9fdd-32a128d85b1a service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:42:59.205535 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eee4a7-6abc-4790-9fdd-32a128d85b1a service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:42:59.205535 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eee4a7-6abc-4790-9fdd-32a128d85b1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:42:59.206960 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-65eee4a7-6abc-4790-9fdd-32a128d85b1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:42:59.215137 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:42:59.304770 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eee4a7-6abc-4790-9fdd-32a128d85b1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:42:59.306436 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eee4a7-6abc-4790-9fdd-32a128d85b1a service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:42:59.324957 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 713/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:00.400452 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57ea7a-a2cf-4e93-9e60-fb5b569e8088 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:00.401037 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57ea7a-a2cf-4e93-9e60-fb5b569e8088 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:00.401037 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57ea7a-a2cf-4e93-9e60-fb5b569e8088 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:00.402277 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 12:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:00.409102 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d52502-4b7a-4a1b-b446-57432f75f81b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:00.409612 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d52502-4b7a-4a1b-b446-57432f75f81b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:00.409737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d52502-4b7a-4a1b-b446-57432f75f81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:00.487723 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c5d52502-4b7a-4a1b-b446-57432f75f81b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:00.488977 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c5d52502-4b7a-4a1b-b446-57432f75f81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:00.561020 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:00.589817 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 12:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:00.739015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57ea7a-a2cf-4e93-9e60-fb5b569e8088 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:00.740462 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-0d57ea7a-a2cf-4e93-9e60-fb5b569e8088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:00.758006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:00.885236 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57ea7a-a2cf-4e93-9e60-fb5b569e8088 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:00.971843 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:00.999216 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a7dfc-3b9a-48be-942f-2a83d4875999 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:00.999735 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a7dfc-3b9a-48be-942f-2a83d4875999 service nova] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:01.000006 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a7dfc-3b9a-48be-942f-2a83d4875999 service nova] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:01.000269 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a7dfc-3b9a-48be-942f-2a83d4875999 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:01.001849 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-a95a7dfc-3b9a-48be-942f-2a83d4875999 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:01.091819 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a7dfc-3b9a-48be-942f-2a83d4875999 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:01.093421 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a7dfc-3b9a-48be-942f-2a83d4875999 service nova] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:01.111752 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 720/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:02.184206 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064331c9-3741-4c1e-8986-491c44eb4eeb None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:02.184768 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064331c9-3741-4c1e-8986-491c44eb4eeb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:02.184768 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064331c9-3741-4c1e-8986-491c44eb4eeb None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:02.186119 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 12:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:02.192530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcaa4c-f571-4d01-a8e9-aa80e2409f0c None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:02.192973 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcaa4c-f571-4d01-a8e9-aa80e2409f0c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:02.193100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcaa4c-f571-4d01-a8e9-aa80e2409f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:02.274024 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-4bbcaa4c-f571-4d01-a8e9-aa80e2409f0c None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:02.275772 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-4bbcaa4c-f571-4d01-a8e9-aa80e2409f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:02.320293 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:02.376774 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 12:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:02.528291 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064331c9-3741-4c1e-8986-491c44eb4eeb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:02.529706 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-064331c9-3741-4c1e-8986-491c44eb4eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:02.677995 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064331c9-3741-4c1e-8986-491c44eb4eeb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:02.732875 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:02.802285 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516565c8-8360-4fb7-9060-11f12608a23b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:02.802815 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516565c8-8360-4fb7-9060-11f12608a23b service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:02.803092 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516565c8-8360-4fb7-9060-11f12608a23b service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:02.803367 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516565c8-8360-4fb7-9060-11f12608a23b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:02.805094 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-516565c8-8360-4fb7-9060-11f12608a23b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:02.910353 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516565c8-8360-4fb7-9060-11f12608a23b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:02.912051 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516565c8-8360-4fb7-9060-11f12608a23b service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:02.932537 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 716/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:03.995917 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7939f-a2bf-4c73-9104-298b7005fe36 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:03.996511 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7939f-a2bf-4c73-9104-298b7005fe36 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:03.996511 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7939f-a2bf-4c73-9104-298b7005fe36 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:03.997788 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 12:43:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:04.004620 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e8e33-14d1-4289-99f9-107cadef5bb8 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:04.005110 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e8e33-14d1-4289-99f9-107cadef5bb8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:04.005110 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e8e33-14d1-4289-99f9-107cadef5bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:04.083392 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1d9e8e33-14d1-4289-99f9-107cadef5bb8 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:04.084591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1d9e8e33-14d1-4289-99f9-107cadef5bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:04.086623 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:04.183436 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 12:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:04.332422 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7939f-a2bf-4c73-9104-298b7005fe36 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:04.333949 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-edc7939f-a2bf-4c73-9104-298b7005fe36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:04.481527 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7939f-a2bf-4c73-9104-298b7005fe36 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:04.508009 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:04.594972 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f2981-fa90-4fc9-a172-0e29daf415c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:04.595488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f2981-fa90-4fc9-a172-0e29daf415c4 service nova] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:04.595765 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f2981-fa90-4fc9-a172-0e29daf415c4 service nova] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:04.596035 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f2981-fa90-4fc9-a172-0e29daf415c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:04.597660 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-f47f2981-fa90-4fc9-a172-0e29daf415c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:04.688257 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f2981-fa90-4fc9-a172-0e29daf415c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:04.689868 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f2981-fa90-4fc9-a172-0e29daf415c4 service nova] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:04.707746 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 723/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:06.810370 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07d8b5-6106-465d-aa73-c2b968b315e9 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:06.810912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07d8b5-6106-465d-aa73-c2b968b315e9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:06.810912 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07d8b5-6106-465d-aa73-c2b968b315e9 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:06.812262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:06.817798 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ed615-c005-4d40-8a63-0a9cc9a73630 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:06.818216 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ed615-c005-4d40-8a63-0a9cc9a73630 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:06.818369 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ed615-c005-4d40-8a63-0a9cc9a73630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:06.860344 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:06.886707 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-fb0ed615-c005-4d40-8a63-0a9cc9a73630 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:06.887909 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-fb0ed615-c005-4d40-8a63-0a9cc9a73630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:06.982268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:07.269119 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07d8b5-6106-465d-aa73-c2b968b315e9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:07.270715 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7f07d8b5-6106-465d-aa73-c2b968b315e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:07.376376 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07d8b5-6106-465d-aa73-c2b968b315e9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:07.410010 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:07.453546 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fe681-de53-4b7e-bb2a-66e6cb242cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:07.454184 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fe681-de53-4b7e-bb2a-66e6cb242cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:07.454458 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fe681-de53-4b7e-bb2a-66e6cb242cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:07.454725 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fe681-de53-4b7e-bb2a-66e6cb242cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:07.456326 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-342fe681-de53-4b7e-bb2a-66e6cb242cb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:07.566787 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342fe681-de53-4b7e-bb2a-66e6cb242cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:07.568165 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342fe681-de53-4b7e-bb2a-66e6cb242cb1 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:07.587969 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 719/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:09.968823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c995d58-2c95-4f0e-a4ed-12ee01024f6d None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:09.969395 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c995d58-2c95-4f0e-a4ed-12ee01024f6d None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:09.969395 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c995d58-2c95-4f0e-a4ed-12ee01024f6d None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:09.970737 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:09.976040 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109b77e-2726-4f40-aafc-d57967a020db None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:09.976497 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109b77e-2726-4f40-aafc-d57967a020db None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:09.976631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109b77e-2726-4f40-aafc-d57967a020db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:10.169327 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e109b77e-2726-4f40-aafc-d57967a020db None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:10.170557 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e109b77e-2726-4f40-aafc-d57967a020db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:10.183417 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:10.271161 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:10.556418 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c995d58-2c95-4f0e-a4ed-12ee01024f6d None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:10.558134 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-0c995d58-2c95-4f0e-a4ed-12ee01024f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:10.660726 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c995d58-2c95-4f0e-a4ed-12ee01024f6d None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:10.674172 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:10.734886 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b091a56-1cc8-4dc1-9872-2c1f10e0001a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:10.735492 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b091a56-1cc8-4dc1-9872-2c1f10e0001a service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:10.735794 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b091a56-1cc8-4dc1-9872-2c1f10e0001a service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:10.736060 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b091a56-1cc8-4dc1-9872-2c1f10e0001a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:10.737696 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-1b091a56-1cc8-4dc1-9872-2c1f10e0001a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:10.849249 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b091a56-1cc8-4dc1-9872-2c1f10e0001a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:10.850675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b091a56-1cc8-4dc1-9872-2c1f10e0001a service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:10.868923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 726/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:13.435093 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d43451-2b6d-456a-8324-2945daa3c782 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:13.435626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d43451-2b6d-456a-8324-2945daa3c782 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:13.435626 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d43451-2b6d-456a-8324-2945daa3c782 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:13.436964 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 12:43:13.442351 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c90fb6-50fd-487a-a97e-b7fb4af24885 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:13.442784 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c90fb6-50fd-487a-a97e-b7fb4af24885 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:13.442946 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c90fb6-50fd-487a-a97e-b7fb4af24885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:13.491582 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:13.512865 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-22c90fb6-50fd-487a-a97e-b7fb4af24885 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:13.513984 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-22c90fb6-50fd-487a-a97e-b7fb4af24885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:13.614171 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:13.904624 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d43451-2b6d-456a-8324-2945daa3c782 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:13.906334 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e3d43451-2b6d-456a-8324-2945daa3c782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:14.017980 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d43451-2b6d-456a-8324-2945daa3c782 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:14.059872 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:14.099182 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e0b52-793d-4466-98c9-a061acf06e2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:14.099793 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e0b52-793d-4466-98c9-a061acf06e2d service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:14.100066 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e0b52-793d-4466-98c9-a061acf06e2d service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:14.100331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e0b52-793d-4466-98c9-a061acf06e2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:14.101963 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ed6e0b52-793d-4466-98c9-a061acf06e2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:14.215814 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e0b52-793d-4466-98c9-a061acf06e2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:14.217156 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e0b52-793d-4466-98c9-a061acf06e2d service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:14.235068 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 722/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:16.531488 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916f032-4328-4fff-848a-950f08dacbbd None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:16.532059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916f032-4328-4fff-848a-950f08dacbbd None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:16.532059 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916f032-4328-4fff-848a-950f08dacbbd None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:16.533353 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:16.538690 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d8613-77ca-41a8-a281-f31980f74b59 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:16.539128 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d8613-77ca-41a8-a281-f31980f74b59 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:16.539224 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d8613-77ca-41a8-a281-f31980f74b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:16.609340 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-cb3d8613-77ca-41a8-a281-f31980f74b59 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:16.610589 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cb3d8613-77ca-41a8-a281-f31980f74b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:16.631318 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:16.708189 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:16.995229 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916f032-4328-4fff-848a-950f08dacbbd None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:16.996809 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-6916f032-4328-4fff-848a-950f08dacbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:17.108684 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916f032-4328-4fff-848a-950f08dacbbd None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:17.129930 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:17.189142 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87daf59b-2ede-4d91-81d2-b115a3afc354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:17.189629 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daf59b-2ede-4d91-81d2-b115a3afc354 service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:17.189935 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daf59b-2ede-4d91-81d2-b115a3afc354 service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:17.190223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daf59b-2ede-4d91-81d2-b115a3afc354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:17.191865 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-87daf59b-2ede-4d91-81d2-b115a3afc354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:17.308112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87daf59b-2ede-4d91-81d2-b115a3afc354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:17.309547 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87daf59b-2ede-4d91-81d2-b115a3afc354 service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:17.327995 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 729/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:19.572804 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61609b1d-421b-429c-8ad9-ff4810220fb2 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:19.573353 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61609b1d-421b-429c-8ad9-ff4810220fb2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:19.573353 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61609b1d-421b-429c-8ad9-ff4810220fb2 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:19.574737 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:19.580160 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad03d97-6fc7-48a8-b4e0-ba9f6c8a3054 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:19.580620 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad03d97-6fc7-48a8-b4e0-ba9f6c8a3054 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:19.580727 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad03d97-6fc7-48a8-b4e0-ba9f6c8a3054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:19.645529 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:19.648818 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-2ad03d97-6fc7-48a8-b4e0-ba9f6c8a3054 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:19.651040 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-2ad03d97-6fc7-48a8-b4e0-ba9f6c8a3054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:19.749508 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:20.039765 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61609b1d-421b-429c-8ad9-ff4810220fb2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:20.041361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-61609b1d-421b-429c-8ad9-ff4810220fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:20.150433 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61609b1d-421b-429c-8ad9-ff4810220fb2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:20.202541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:20.227394 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6209f-de4f-4ed6-bae5-5f596134b0fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:20.228123 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6209f-de4f-4ed6-bae5-5f596134b0fa service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:20.228595 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6209f-de4f-4ed6-bae5-5f596134b0fa service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:20.229067 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6209f-de4f-4ed6-bae5-5f596134b0fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:20.232166 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3ba6209f-de4f-4ed6-bae5-5f596134b0fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:20.345061 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6209f-de4f-4ed6-bae5-5f596134b0fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:20.346511 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6209f-de4f-4ed6-bae5-5f596134b0fa service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:20.364401 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 725/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:22.648910 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be7c2f-4264-4d88-94f7-5edee7f50cb8 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:22.649499 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be7c2f-4264-4d88-94f7-5edee7f50cb8 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:22.649499 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be7c2f-4264-4d88-94f7-5edee7f50cb8 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:22.651307 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:22.656664 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7f67c-566a-48dc-b0bf-c08a0eebdef4 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:22.657108 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7f67c-566a-48dc-b0bf-c08a0eebdef4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:22.657230 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7f67c-566a-48dc-b0bf-c08a0eebdef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:22.727500 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9bf7f67c-566a-48dc-b0bf-c08a0eebdef4 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:22.729375 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9bf7f67c-566a-48dc-b0bf-c08a0eebdef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:22.758316 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:22.829521 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:23.114848 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be7c2f-4264-4d88-94f7-5edee7f50cb8 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:23.116395 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-45be7c2f-4264-4d88-94f7-5edee7f50cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:23.225093 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be7c2f-4264-4d88-94f7-5edee7f50cb8 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:23.263616 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:23.306643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0689c-b3c0-4432-8899-fb2cf624e220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:23.307241 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0689c-b3c0-4432-8899-fb2cf624e220 service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:23.307514 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0689c-b3c0-4432-8899-fb2cf624e220 service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:23.307779 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0689c-b3c0-4432-8899-fb2cf624e220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:23.309370 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-ccc0689c-b3c0-4432-8899-fb2cf624e220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:23.419391 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0689c-b3c0-4432-8899-fb2cf624e220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:23.420754 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0689c-b3c0-4432-8899-fb2cf624e220 service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:23.439086 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 732/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:25.697429 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c8572-06ef-414e-9d64-ac475bf3b44a None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:25.697978 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c8572-06ef-414e-9d64-ac475bf3b44a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:25.697978 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c8572-06ef-414e-9d64-ac475bf3b44a None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:25.699331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:43:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:25.704662 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b15f1-15a5-4fdb-8f1f-01617e467a17 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:25.705126 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b15f1-15a5-4fdb-8f1f-01617e467a17 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:25.705223 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b15f1-15a5-4fdb-8f1f-01617e467a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:25.772540 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-7f0b15f1-15a5-4fdb-8f1f-01617e467a17 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:25.773767 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-7f0b15f1-15a5-4fdb-8f1f-01617e467a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:25.782341 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:25.872517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:26.157557 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c8572-06ef-414e-9d64-ac475bf3b44a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:26.159162 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-470c8572-06ef-414e-9d64-ac475bf3b44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:26.270325 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c8572-06ef-414e-9d64-ac475bf3b44a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:26.326921 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:26.345561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e2b10-def2-4dc9-aabc-6de05bad26e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:26.346203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e2b10-def2-4dc9-aabc-6de05bad26e6 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:26.346482 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e2b10-def2-4dc9-aabc-6de05bad26e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:26.346752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e2b10-def2-4dc9-aabc-6de05bad26e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:26.348350 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-775e2b10-def2-4dc9-aabc-6de05bad26e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:26.460943 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e2b10-def2-4dc9-aabc-6de05bad26e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:26.462958 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e2b10-def2-4dc9-aabc-6de05bad26e6 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:26.481209 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 728/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:28.726908 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b2f26-71eb-4c50-9213-f3b268af1925 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:28.727485 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b2f26-71eb-4c50-9213-f3b268af1925 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:28.727485 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b2f26-71eb-4c50-9213-f3b268af1925 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:28.729392 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:28.734895 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd11415-367f-445b-99a7-66a6d2a963b4 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:28.735363 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd11415-367f-445b-99a7-66a6d2a963b4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:28.735467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd11415-367f-445b-99a7-66a6d2a963b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:28.803351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-7fd11415-367f-445b-99a7-66a6d2a963b4 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:28.804573 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7fd11415-367f-445b-99a7-66a6d2a963b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:28.844565 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:28.905477 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:29.188880 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b2f26-71eb-4c50-9213-f3b268af1925 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:29.190478 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-4d5b2f26-71eb-4c50-9213-f3b268af1925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:29.298177 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b2f26-71eb-4c50-9213-f3b268af1925 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:29.336037 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:29.375261 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d48fd-1816-4235-9154-1a0732e1c896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:29.375878 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d48fd-1816-4235-9154-1a0732e1c896 service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:29.376153 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d48fd-1816-4235-9154-1a0732e1c896 service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:29.376422 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d48fd-1816-4235-9154-1a0732e1c896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:29.378061 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-e99d48fd-1816-4235-9154-1a0732e1c896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:29.490851 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d48fd-1816-4235-9154-1a0732e1c896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:29.492238 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d48fd-1816-4235-9154-1a0732e1c896 service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:29.510544 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 735/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:31.728916 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e806d-578e-48df-bc68-c57495f3a304 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:31.729510 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e806d-578e-48df-bc68-c57495f3a304 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:31.729510 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e806d-578e-48df-bc68-c57495f3a304 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:31.730813 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:31.736216 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2c354-8dee-4710-bf40-7d328f9a9d72 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:31.736711 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2c354-8dee-4710-bf40-7d328f9a9d72 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:31.736817 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2c354-8dee-4710-bf40-7d328f9a9d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:31.803466 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-48c2c354-8dee-4710-bf40-7d328f9a9d72 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:31.804675 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-48c2c354-8dee-4710-bf40-7d328f9a9d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:31.823083 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:31.902875 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:32.187705 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e806d-578e-48df-bc68-c57495f3a304 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:32.189239 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f05e806d-578e-48df-bc68-c57495f3a304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:32.297932 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e806d-578e-48df-bc68-c57495f3a304 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:32.366173 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:32.374166 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af4c4e-f57a-4618-b7f9-79932f0bcb9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:32.374773 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af4c4e-f57a-4618-b7f9-79932f0bcb9e service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:32.375047 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af4c4e-f57a-4618-b7f9-79932f0bcb9e service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:32.375318 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af4c4e-f57a-4618-b7f9-79932f0bcb9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:32.376918 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-38af4c4e-f57a-4618-b7f9-79932f0bcb9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:32.490561 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af4c4e-f57a-4618-b7f9-79932f0bcb9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:32.492080 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af4c4e-f57a-4618-b7f9-79932f0bcb9e service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:32.510445 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 731/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:34.693549 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a76a1b-fc67-40eb-9eb4-cbf0dab2f251 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:34.694154 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a76a1b-fc67-40eb-9eb4-cbf0dab2f251 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:34.694154 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a76a1b-fc67-40eb-9eb4-cbf0dab2f251 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:34.695441 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 12:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:34.700840 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada568-fdea-4f92-8dab-2562abb3f7c3 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:34.701262 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada568-fdea-4f92-8dab-2562abb3f7c3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:34.701387 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada568-fdea-4f92-8dab-2562abb3f7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:34.766891 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3fada568-fdea-4f92-8dab-2562abb3f7c3 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:34.768099 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3fada568-fdea-4f92-8dab-2562abb3f7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:34.817187 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:34.866331 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 12:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:35.148539 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a76a1b-fc67-40eb-9eb4-cbf0dab2f251 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:35.150740 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-a4a76a1b-fc67-40eb-9eb4-cbf0dab2f251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:35.262530 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a76a1b-fc67-40eb-9eb4-cbf0dab2f251 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:35.315308 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:35.342836 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39264c20-00ad-4dbe-a70f-4e20d1eadbec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:35.343435 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39264c20-00ad-4dbe-a70f-4e20d1eadbec service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:35.343704 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39264c20-00ad-4dbe-a70f-4e20d1eadbec service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:35.343967 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39264c20-00ad-4dbe-a70f-4e20d1eadbec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:35.345591 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-39264c20-00ad-4dbe-a70f-4e20d1eadbec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:35.457799 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39264c20-00ad-4dbe-a70f-4e20d1eadbec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:35.459211 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39264c20-00ad-4dbe-a70f-4e20d1eadbec service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:35.476681 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 738/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:44.864756 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=22 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:45.474550 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=66 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:55.577875 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb346b1e-05b6-45c0-b353-768eb60845a9 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:55.578483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb346b1e-05b6-45c0-b353-768eb60845a9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:55.578483 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb346b1e-05b6-45c0-b353-768eb60845a9 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:55.579786 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 733/1471] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:55.585169 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0a4c0-f87f-4765-97d1-5dae48b9bb8b None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:55.585643 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0a4c0-f87f-4765-97d1-5dae48b9bb8b None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:55.585753 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0a4c0-f87f-4765-97d1-5dae48b9bb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:55.651702 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-80b0a4c0-f87f-4765-97d1-5dae48b9bb8b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:55.652947 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-80b0a4c0-f87f-4765-97d1-5dae48b9bb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:55.747155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 739/1472] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:56.054625 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d224c3-ebff-48c9-888a-efcc2a1aa025 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:56.054899 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d224c3-ebff-48c9-888a-efcc2a1aa025 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:56.055188 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d224c3-ebff-48c9-888a-efcc2a1aa025 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:56.057087 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 734/1473] 10.222.0.33 () {54 vars in 962 bytes} [Tue Oct 4 12:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:56.062413 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8da401-4bd4-46ab-92b7-91c6c4e74f7e None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:56.062763 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8da401-4bd4-46ab-92b7-91c6c4e74f7e None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:56.063156 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8da401-4bd4-46ab-92b7-91c6c4e74f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:56.141450 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-ec8da401-4bd4-46ab-92b7-91c6c4e74f7e None None] MFA Rules not processed for user `f2f4f9541f89433cad52e826794b049f`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:56.142761 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-ec8da401-4bd4-46ab-92b7-91c6c4e74f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:56.163602 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:56.220565 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 740/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Tue Oct 4 12:43:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:56.229230 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d224c3-ebff-48c9-888a-efcc2a1aa025 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:56.230785 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-96d224c3-ebff-48c9-888a-efcc2a1aa025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:56.336067 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d224c3-ebff-48c9-888a-efcc2a1aa025 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:56.417373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a1d5d-e042-42cb-b80e-7f1324cfc9b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f4f9541f89433cad52e826794b049f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:56.418007 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a1d5d-e042-42cb-b80e-7f1324cfc9b5 service glance] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:56.418312 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a1d5d-e042-42cb-b80e-7f1324cfc9b5 service glance] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:56.418584 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a1d5d-e042-42cb-b80e-7f1324cfc9b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:56.420219 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a81a1d5d-e042-42cb-b80e-7f1324cfc9b5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:56.475809 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:56.533575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a1d5d-e042-42cb-b80e-7f1324cfc9b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:56.535054 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a1d5d-e042-42cb-b80e-7f1324cfc9b5 service glance] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:56.554422 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 735/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:43:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:43:58.826445 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d56098-a958-455e-95c3-9ff8edfa6457 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:58.827069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d56098-a958-455e-95c3-9ff8edfa6457 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:58.827069 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d56098-a958-455e-95c3-9ff8edfa6457 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:58.828370 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 741/1476] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:43:58.833734 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1bb37-e997-47d1-9817-d424a76ff459 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:58.834201 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1bb37-e997-47d1-9817-d424a76ff459 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:58.834361 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1bb37-e997-47d1-9817-d424a76ff459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:58.900554 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-78b1bb37-e997-47d1-9817-d424a76ff459 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:43:58.901773 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-78b1bb37-e997-47d1-9817-d424a76ff459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:58.998762 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 736/1477] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:43:59.276194 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d56098-a958-455e-95c3-9ff8edfa6457 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:43:59.277791 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-05d56098-a958-455e-95c3-9ff8edfa6457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:43:59.404846 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d56098-a958-455e-95c3-9ff8edfa6457 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:43:59.461078 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:43:59.506707 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c5e6b-dcb2-4923-88ef-c7d84c346af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:43:59.507319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c5e6b-dcb2-4923-88ef-c7d84c346af4 admin admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:43:59.507598 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c5e6b-dcb2-4923-88ef-c7d84c346af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:43:59.507865 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c5e6b-dcb2-4923-88ef-c7d84c346af4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:43:59.509033 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c5e6b-dcb2-4923-88ef-c7d84c346af4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:43:59.511128 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: WARNING py.warnings [None req-fe2c5e6b-dcb2-4923-88ef-c7d84c346af4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:43:59.511128 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: warnings.warn(msg) Oct 04 12:43:59.511128 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: Oct 04 12:43:59.512225 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c5e6b-dcb2-4923-88ef-c7d84c346af4 admin admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:43:59.531772 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 742/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Tue Oct 4 12:43:59 2022] GET /identity/v3/projects => generated 2553 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:44:01.746978 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf61ec-71fb-4432-af72-c779c0cc3774 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:01.747503 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf61ec-71fb-4432-af72-c779c0cc3774 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:01.747503 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf61ec-71fb-4432-af72-c779c0cc3774 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:01.748828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 737/1479] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:44:01.754348 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eb8d1-2557-49bf-a33c-7f0077af5e01 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:01.754792 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eb8d1-2557-49bf-a33c-7f0077af5e01 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:01.754895 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eb8d1-2557-49bf-a33c-7f0077af5e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:01.822268 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-f80eb8d1-2557-49bf-a33c-7f0077af5e01 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:44:01.823479 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-f80eb8d1-2557-49bf-a33c-7f0077af5e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:01.920980 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 743/1480] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:44:02.217577 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf61ec-71fb-4432-af72-c779c0cc3774 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:44:02.219273 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dddf61ec-71fb-4432-af72-c779c0cc3774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:02.229294 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:44:02.371027 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf61ec-71fb-4432-af72-c779c0cc3774 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:44:02.451588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:44:02.484588 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35439eb-c940-48df-b2fc-5d503804e460 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02260480264b4f8f8a7e5cb647ac5d9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:44:02.485194 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35439eb-c940-48df-b2fc-5d503804e460 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:02.485467 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35439eb-c940-48df-b2fc-5d503804e460 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:02.485733 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35439eb-c940-48df-b2fc-5d503804e460 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:02.487351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e35439eb-c940-48df-b2fc-5d503804e460 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:02.579197 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35439eb-c940-48df-b2fc-5d503804e460 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:44:02.580802 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35439eb-c940-48df-b2fc-5d503804e460 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:44:02.600909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 738/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:44:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:44:05.003986 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4839ec7-1437-4dce-94c5-da6b50f4237c None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:05.004579 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4839ec7-1437-4dce-94c5-da6b50f4237c None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:05.004579 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4839ec7-1437-4dce-94c5-da6b50f4237c None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:05.005849 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 744/1482] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:44:05.012741 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b9bcc-2cd1-4ea5-8111-9221e3372980 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:05.013278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b9bcc-2cd1-4ea5-8111-9221e3372980 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:05.013278 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b9bcc-2cd1-4ea5-8111-9221e3372980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:05.083885 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3f1b9bcc-2cd1-4ea5-8111-9221e3372980 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:44:05.085861 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3f1b9bcc-2cd1-4ea5-8111-9221e3372980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:05.138464 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:44:05.199514 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 739/1483] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:44:05.493923 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4839ec7-1437-4dce-94c5-da6b50f4237c None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:44:05.495517 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-f4839ec7-1437-4dce-94c5-da6b50f4237c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:05.520944 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:44:05.607167 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4839ec7-1437-4dce-94c5-da6b50f4237c None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:44:05.685579 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2681d-44ea-40f7-9deb-b530e5a1c9aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:44:05.686218 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2681d-44ea-40f7-9deb-b530e5a1c9aa service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:05.686491 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2681d-44ea-40f7-9deb-b530e5a1c9aa service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:05.686749 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2681d-44ea-40f7-9deb-b530e5a1c9aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:05.688387 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-eaa2681d-44ea-40f7-9deb-b530e5a1c9aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:05.773378 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:44:05.814246 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2681d-44ea-40f7-9deb-b530e5a1c9aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:44:05.815725 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2681d-44ea-40f7-9deb-b530e5a1c9aa service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:44:05.838132 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 745/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:44:08.111059 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294af2d-7a7d-428d-a88c-267942f49ae4 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:08.111506 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294af2d-7a7d-428d-a88c-267942f49ae4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:08.111619 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294af2d-7a7d-428d-a88c-267942f49ae4 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:08.112917 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 740/1485] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:44:08.118313 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff39ce7-4f0d-4cd7-8332-fa37a830c088 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:08.118745 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff39ce7-4f0d-4cd7-8332-fa37a830c088 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:08.118859 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff39ce7-4f0d-4cd7-8332-fa37a830c088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:08.186296 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-5ff39ce7-4f0d-4cd7-8332-fa37a830c088 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:44:08.187543 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-5ff39ce7-4f0d-4cd7-8332-fa37a830c088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:08.286015 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 746/1486] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:44:08.581035 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294af2d-7a7d-428d-a88c-267942f49ae4 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:44:08.582593 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d294af2d-7a7d-428d-a88c-267942f49ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:08.690125 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294af2d-7a7d-428d-a88c-267942f49ae4 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:44:08.763194 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:44:08.764933 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66b6e5-2f60-486c-a560-0a3e9e0bdd0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:44:08.765509 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66b6e5-2f60-486c-a560-0a3e9e0bdd0f service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:08.765994 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66b6e5-2f60-486c-a560-0a3e9e0bdd0f service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:08.766333 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66b6e5-2f60-486c-a560-0a3e9e0bdd0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:08.768274 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5a66b6e5-2f60-486c-a560-0a3e9e0bdd0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:08.878703 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66b6e5-2f60-486c-a560-0a3e9e0bdd0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:44:08.880135 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66b6e5-2f60-486c-a560-0a3e9e0bdd0f service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:44:08.898887 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 741/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:44:11.251464 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953c1eb-1274-4bc6-a7d6-83d20f79c601 None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:11.252014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953c1eb-1274-4bc6-a7d6-83d20f79c601 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:11.252014 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953c1eb-1274-4bc6-a7d6-83d20f79c601 None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:11.253398 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 747/1488] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:44:11.260197 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3574f0a-deec-4386-8eb2-920c068fb3f8 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:11.260575 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3574f0a-deec-4386-8eb2-920c068fb3f8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:11.260673 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3574f0a-deec-4386-8eb2-920c068fb3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:11.330014 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b3574f0a-deec-4386-8eb2-920c068fb3f8 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:44:11.331287 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b3574f0a-deec-4386-8eb2-920c068fb3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:11.384065 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:44:11.432175 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 742/1489] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:44:11.723286 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953c1eb-1274-4bc6-a7d6-83d20f79c601 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:44:11.724838 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-7953c1eb-1274-4bc6-a7d6-83d20f79c601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:11.763538 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:44:11.831369 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953c1eb-1274-4bc6-a7d6-83d20f79c601 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:44:11.904449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74206fd6-7d87-4372-9bb2-e15eaf0a0f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:44:11.905056 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74206fd6-7d87-4372-9bb2-e15eaf0a0f26 service neutron] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:11.905331 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74206fd6-7d87-4372-9bb2-e15eaf0a0f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:11.905597 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74206fd6-7d87-4372-9bb2-e15eaf0a0f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:11.907231 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-74206fd6-7d87-4372-9bb2-e15eaf0a0f26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:11.986953 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:44:12.020787 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74206fd6-7d87-4372-9bb2-e15eaf0a0f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:44:12.022302 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74206fd6-7d87-4372-9bb2-e15eaf0a0f26 service neutron] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:44:12.040074 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 748/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:44:14.456591 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6929809-e553-4e11-92c3-de213b701f74 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:14.457247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6929809-e553-4e11-92c3-de213b701f74 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:14.457247 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6929809-e553-4e11-92c3-de213b701f74 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:14.458490 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 743/1491] 10.222.0.33 () {54 vars in 927 bytes} [Tue Oct 4 12:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:44:14.464894 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e84877-d9b1-4431-879a-4afdbf8270bc None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:14.465339 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e84877-d9b1-4431-879a-4afdbf8270bc None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:14.465449 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e84877-d9b1-4431-879a-4afdbf8270bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:14.534115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-20e84877-d9b1-4431-879a-4afdbf8270bc None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:44:14.535319 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-20e84877-d9b1-4431-879a-4afdbf8270bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:14.631573 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 749/1492] 10.222.0.33 () {58 vars in 1024 bytes} [Tue Oct 4 12:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:44:14.927351 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6929809-e553-4e11-92c3-de213b701f74 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:44:14.928953 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b6929809-e553-4e11-92c3-de213b701f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:15.033631 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6929809-e553-4e11-92c3-de213b701f74 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:44:15.107663 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd0cc8-26f7-4a6e-a842-01aafa2f6680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:44:15.108268 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd0cc8-26f7-4a6e-a842-01aafa2f6680 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:44:15.108542 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd0cc8-26f7-4a6e-a842-01aafa2f6680 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:44:15.108809 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd0cc8-26f7-4a6e-a842-01aafa2f6680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:44:15.110472 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-fdbd0cc8-26f7-4a6e-a842-01aafa2f6680 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:44:15.120752 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:44:15.225510 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd0cc8-26f7-4a6e-a842-01aafa2f6680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:44:15.227003 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd0cc8-26f7-4a6e-a842-01aafa2f6680 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:44:15.245369 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 744/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:44:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:44:24.629972 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=82 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:44:25.243117 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=50 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:45:28.601477 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6392d-e3c9-4b37-9edc-5d07e0661084 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:28.602110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6392d-e3c9-4b37-9edc-5d07e0661084 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:28.602110 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6392d-e3c9-4b37-9edc-5d07e0661084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:28.679909 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-18b6392d-e3c9-4b37-9edc-5d07e0661084 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:45:28.681112 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-18b6392d-e3c9-4b37-9edc-5d07e0661084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:28.782099 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 750/1494] 10.222.0.33 () {58 vars in 958 bytes} [Tue Oct 4 12:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:45:28.790642 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e3dad-cfdf-48d7-b2dc-229cc052fd2b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:28.791153 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e3dad-cfdf-48d7-b2dc-229cc052fd2b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:28.791249 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e3dad-cfdf-48d7-b2dc-229cc052fd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:28.867893 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-668e3dad-cfdf-48d7-b2dc-229cc052fd2b None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:45:28.869155 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-668e3dad-cfdf-48d7-b2dc-229cc052fd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:28.968689 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 745/1495] 10.222.0.33 () {58 vars in 958 bytes} [Tue Oct 4 12:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:45:28.973322 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6392d-e3c9-4b37-9edc-5d07e0661084 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:45:28.974541 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-18b6392d-e3c9-4b37-9edc-5d07e0661084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:28.985883 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6392d-e3c9-4b37-9edc-5d07e0661084 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:45:29.054973 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:45:29.104113 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e99bc2-777e-4c2d-82d8-056b7b4f404f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:45:29.105143 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99bc2-777e-4c2d-82d8-056b7b4f404f admin admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:29.105631 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99bc2-777e-4c2d-82d8-056b7b4f404f admin admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:29.106143 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99bc2-777e-4c2d-82d8-056b7b4f404f admin admin] PATH_INFO: `/v3/domains` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:29.107324 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e99bc2-777e-4c2d-82d8-056b7b4f404f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 12:45:29.107856 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e99bc2-777e-4c2d-82d8-056b7b4f404f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:45:29.109716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: WARNING py.warnings [None req-92e99bc2-777e-4c2d-82d8-056b7b4f404f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:45:29.109716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: warnings.warn(msg) Oct 04 12:45:29.109716 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: Oct 04 12:45:29.111411 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e99bc2-777e-4c2d-82d8-056b7b4f404f admin admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:45:29.127888 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 751/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Tue Oct 4 12:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 12:45:29.137056 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e3dad-cfdf-48d7-b2dc-229cc052fd2b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:45:29.146512 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e3dad-cfdf-48d7-b2dc-229cc052fd2b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:45:29.210726 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:45:29.246547 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b77da1-c6e0-417c-87e6-81f905f87bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:45:29.247163 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b77da1-c6e0-417c-87e6-81f905f87bdd admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:29.247437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b77da1-c6e0-417c-87e6-81f905f87bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:29.247703 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b77da1-c6e0-417c-87e6-81f905f87bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:29.248344 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b77da1-c6e0-417c-87e6-81f905f87bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:45:29.250144 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: WARNING py.warnings [None req-d7b77da1-c6e0-417c-87e6-81f905f87bdd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:45:29.250144 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: warnings.warn(msg) Oct 04 12:45:29.250144 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: Oct 04 12:45:29.251373 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b77da1-c6e0-417c-87e6-81f905f87bdd admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:45:29.310573 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 746/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Tue Oct 4 12:45:29 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:45:29.314688 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e99bc2-777e-4c2d-82d8-056b7b4f404f admin admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:45:29.316211 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-92e99bc2-777e-4c2d-82d8-056b7b4f404f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:29.350410 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e99bc2-777e-4c2d-82d8-056b7b4f404f admin admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:45:29.450479 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c5543-d0e4-458f-8a89-65adfd19c61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:45:29.451090 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c5543-d0e4-458f-8a89-65adfd19c61f admin admin] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:29.451365 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c5543-d0e4-458f-8a89-65adfd19c61f admin admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:29.451631 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c5543-d0e4-458f-8a89-65adfd19c61f admin admin] PATH_INFO: `/v3/users` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:29.452266 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c5543-d0e4-458f-8a89-65adfd19c61f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:45:29.453276 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: WARNING py.warnings [None req-353c5543-d0e4-458f-8a89-65adfd19c61f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:45:29.453276 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: warnings.warn(msg) Oct 04 12:45:29.453276 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: Oct 04 12:45:29.454528 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c5543-d0e4-458f-8a89-65adfd19c61f admin admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:45:29.497465 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 752/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Tue Oct 4 12:45:29 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 12:45:29.501403 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b77da1-c6e0-417c-87e6-81f905f87bdd admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:45:29.502961 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d7b77da1-c6e0-417c-87e6-81f905f87bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:29.570911 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b77da1-c6e0-417c-87e6-81f905f87bdd admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:45:29.660437 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:45:29.672491 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a56c9-7fbb-4e11-8868-a9bbc3620d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:45:29.673102 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a56c9-7fbb-4e11-8868-a9bbc3620d0e admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:29.673376 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a56c9-7fbb-4e11-8868-a9bbc3620d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:29.673644 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a56c9-7fbb-4e11-8868-a9bbc3620d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:29.674315 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a56c9-7fbb-4e11-8868-a9bbc3620d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:45:29.675330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: WARNING py.warnings [None req-d56a56c9-7fbb-4e11-8868-a9bbc3620d0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:45:29.675330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: warnings.warn(msg) Oct 04 12:45:29.675330 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: Oct 04 12:45:29.676220 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a56c9-7fbb-4e11-8868-a9bbc3620d0e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:45:29.690789 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 747/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Tue Oct 4 12:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 12:45:29.696621 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c5543-d0e4-458f-8a89-65adfd19c61f admin admin] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:45:29.698187 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-353c5543-d0e4-458f-8a89-65adfd19c61f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:29.710256 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c5543-d0e4-458f-8a89-65adfd19c61f admin admin] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:45:29.735115 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:45:29.805993 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcf824-e655-404b-990c-31fc8a10d6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:45:29.806634 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcf824-e655-404b-990c-31fc8a10d6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:29.806904 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcf824-e655-404b-990c-31fc8a10d6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:29.807170 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcf824-e655-404b-990c-31fc8a10d6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:29.807799 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcf824-e655-404b-990c-31fc8a10d6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:45:29.808823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: WARNING py.warnings [None req-00fcf824-e655-404b-990c-31fc8a10d6b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:45:29.808823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: warnings.warn(msg) Oct 04 12:45:29.808823 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: Oct 04 12:45:29.809708 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcf824-e655-404b-990c-31fc8a10d6b4 admin admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:45:29.822742 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 753/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Tue Oct 4 12:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 12:45:29.826909 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a56c9-7fbb-4e11-8868-a9bbc3620d0e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:45:29.828476 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d56a56c9-7fbb-4e11-8868-a9bbc3620d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:29.862391 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a56c9-7fbb-4e11-8868-a9bbc3620d0e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:45:29.959221 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673a537-75cf-41f6-86a0-4e340652afa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:45:29.960393 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673a537-75cf-41f6-86a0-4e340652afa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:29.960673 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673a537-75cf-41f6-86a0-4e340652afa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:29.960945 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673a537-75cf-41f6-86a0-4e340652afa5 admin admin] PATH_INFO: `/v3/projects/53eefbdc480243dd9234471bd8fe9d87/users/edb73d8759ab495c8c1364178513f34c/roles/d18e4897bf3648498a472a2ea8174f8b` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:29.985153 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673a537-75cf-41f6-86a0-4e340652afa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53eefbdc480243dd9234471bd8fe9d87, user_id=edb73d8759ab495c8c1364178513f34c, role_id=d18e4897bf3648498a472a2ea8174f8b)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:45:29.986218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: WARNING py.warnings [None req-3673a537-75cf-41f6-86a0-4e340652afa5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:45:29.986218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: warnings.warn(msg) Oct 04 12:45:29.986218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: Oct 04 12:45:29.988198 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673a537-75cf-41f6-86a0-4e340652afa5 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:45:30.014632 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 748/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Tue Oct 4 12:45:29 2022] PUT /identity/v3/projects/53eefbdc480243dd9234471bd8fe9d87/users/edb73d8759ab495c8c1364178513f34c/roles/d18e4897bf3648498a472a2ea8174f8b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:45:30.027100 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a89c2-b13a-4c97-a958-707c8a2eed98 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:30.027381 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a89c2-b13a-4c97-a958-707c8a2eed98 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:30.027652 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a89c2-b13a-4c97-a958-707c8a2eed98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:30.089155 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-a93a89c2-b13a-4c97-a958-707c8a2eed98 None None] MFA Rules not processed for user `edb73d8759ab495c8c1364178513f34c`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:45:30.090501 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-a93a89c2-b13a-4c97-a958-707c8a2eed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:30.168868 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 754/1502] 10.222.0.33 () {58 vars in 958 bytes} [Tue Oct 4 12:45:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 12:45:30.182649 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdca31-e5e2-46e3-8743-59b7a19976af None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:30.182929 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdca31-e5e2-46e3-8743-59b7a19976af None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:30.183203 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdca31-e5e2-46e3-8743-59b7a19976af None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:30.184541 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 749/1503] 10.222.0.33 () {54 vars in 963 bytes} [Tue Oct 4 12:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:45:30.189918 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625ea1f-0022-48f9-9e82-b22c7602a1b7 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:30.190238 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625ea1f-0022-48f9-9e82-b22c7602a1b7 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:30.190506 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625ea1f-0022-48f9-9e82-b22c7602a1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:30.224251 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:45:30.270982 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-b625ea1f-0022-48f9-9e82-b22c7602a1b7 None None] MFA Rules not processed for user `b131edeaa17e4e10af385339d463a512`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:45:30.272813 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-b625ea1f-0022-48f9-9e82-b22c7602a1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:30.353435 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 755/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Tue Oct 4 12:45:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:45:30.361218 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdca31-e5e2-46e3-8743-59b7a19976af None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:45:30.362796 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-25fdca31-e5e2-46e3-8743-59b7a19976af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:30.415018 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:45:30.472055 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdca31-e5e2-46e3-8743-59b7a19976af None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:45:30.545889 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fcdb0-c13e-4568-9401-00988e4e7462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b131edeaa17e4e10af385339d463a512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:45:30.546544 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fcdb0-c13e-4568-9401-00988e4e7462 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:30.546817 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fcdb0-c13e-4568-9401-00988e4e7462 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:30.547085 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fcdb0-c13e-4568-9401-00988e4e7462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:30.548838 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-759fcdb0-c13e-4568-9401-00988e4e7462 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:30.648421 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fcdb0-c13e-4568-9401-00988e4e7462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:45:30.649767 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fcdb0-c13e-4568-9401-00988e4e7462 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:45:30.653327 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:45:30.668086 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 750/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:45:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:45:30.713889 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c98fa0-d3ea-46dc-9f67-7707c10cc0fe None None] REQUEST_METHOD: `GET` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:30.714322 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c98fa0-d3ea-46dc-9f67-7707c10cc0fe None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:30.714475 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c98fa0-d3ea-46dc-9f67-7707c10cc0fe None None] PATH_INFO: `/` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:30.715749 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 756/1506] 10.222.0.33 () {52 vars in 830 bytes} [Tue Oct 4 12:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 12:45:30.719485 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fcdb0-c13e-4568-9401-00988e4e7462 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:45:30.720696 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-759fcdb0-c13e-4568-9401-00988e4e7462 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:30.820362 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fcdb0-c13e-4568-9401-00988e4e7462 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:45:30.912694 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:45:30.913372 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:30.913495 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:30.913761 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] PATH_INFO: `/v3/users/edb73d8759ab495c8c1364178513f34c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:30.927828 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb73d8759ab495c8c1364178513f34c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:45:30.929375 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: WARNING py.warnings [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:45:30.929375 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: warnings.warn(msg) Oct 04 12:45:30.929375 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: Oct 04 12:45:30.930684 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:45:30.940410 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:45:30.997116 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb73d8759ab495c8c1364178513f34c'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:45:30.997637 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb73d8759ab495c8c1364178513f34c'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:45:31.015563 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb73d8759ab495c8c1364178513f34c'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:45:31.016029 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb73d8759ab495c8c1364178513f34c'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:45:31.040516 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb73d8759ab495c8c1364178513f34c'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:45:31.055921 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 751/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Tue Oct 4 12:45:30 2022] DELETE /identity/v3/users/edb73d8759ab495c8c1364178513f34c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:45:31.060334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daf0a9-a09c-418a-998f-e30e24e2ae49 None None] REQUEST_METHOD: `POST` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:31.060618 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daf0a9-a09c-418a-998f-e30e24e2ae49 None None] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:31.060885 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daf0a9-a09c-418a-998f-e30e24e2ae49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:31.117072 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.auth.core [None req-32daf0a9-a09c-418a-998f-e30e24e2ae49 None None] MFA Rules not processed for user `ba3be3b6260847d081e4ed3e9d15d818`. Rule list: `[]` (Enabled: `True`). {{(pid=93092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 12:45:31.118294 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-32daf0a9-a09c-418a-998f-e30e24e2ae49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:31.192326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=100 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:45:31.206350 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 757/1508] 10.222.0.33 () {58 vars in 958 bytes} [Tue Oct 4 12:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 12:45:31.215926 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:45:31.217163 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:31.323345 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05152f8f-3fe7-483e-9b2d-c9bab9474995 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:45:31.388125 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:45:31.408699 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164db262-b8a2-4eaf-93ef-4a86b888b26b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '756aec42473f4bebb15f8d43fe16504d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a7fb003a695490faa0077bbbe5e7d77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:45:31.409321 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164db262-b8a2-4eaf-93ef-4a86b888b26b service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:31.409595 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164db262-b8a2-4eaf-93ef-4a86b888b26b service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:31.409862 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164db262-b8a2-4eaf-93ef-4a86b888b26b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:31.411535 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-164db262-b8a2-4eaf-93ef-4a86b888b26b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:31.523491 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164db262-b8a2-4eaf-93ef-4a86b888b26b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:45:31.524942 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164db262-b8a2-4eaf-93ef-4a86b888b26b service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:45:31.543603 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 752/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Tue Oct 4 12:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 12:45:31.903035 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daf0a9-a09c-418a-998f-e30e24e2ae49 None None] Authenticating user token {{(pid=93092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 12:45:31.904334 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.fernet_utils [None req-32daf0a9-a09c-418a-998f-e30e24e2ae49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 12:45:31.916309 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32daf0a9-a09c-418a-998f-e30e24e2ae49 None None] Validating token access rules against request {{(pid=93092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 12:45:32.021326 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba3be3b6260847d081e4ed3e9d15d818', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f85b0de3065741e781a91cbb56d099f5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 12:45:32.022007 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 12:45:32.022813 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 12:45:32.023089 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] PATH_INFO: `/v3/projects/53eefbdc480243dd9234471bd8fe9d87` {{(pid=93092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 12:45:32.033340 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53eefbdc480243dd9234471bd8fe9d87)` {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 12:45:32.034321 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: WARNING py.warnings [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 12:45:32.034321 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: warnings.warn(msg) Oct 04 12:45:32.034321 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: Oct 04 12:45:32.035538 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] RBAC: Authorization granted {{(pid=93092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 12:45:32.094140 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53eefbdc480243dd9234471bd8fe9d87', 'request_id': 'req-ac00bb39-9e0d-404e-a478-33a682cbbb0f'} {{(pid=93092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:45:32.103495 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53eefbdc480243dd9234471bd8fe9d87', 'request_id': 'req-ac00bb39-9e0d-404e-a478-33a682cbbb0f'} {{(pid=93092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:45:32.109603 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53eefbdc480243dd9234471bd8fe9d87', 'request_id': 'req-ac00bb39-9e0d-404e-a478-33a682cbbb0f'} {{(pid=93092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:45:32.140238 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] The token cache is being invalidate because project 53eefbdc480243dd9234471bd8fe9d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 12:45:32.140523 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:45:32.140962 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG keystone.notifications [None req-ac00bb39-9e0d-404e-a478-33a682cbbb0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 12:45:32.142425 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: [pid: 93092|app: 0|req: 758/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Tue Oct 4 12:45:31 2022] DELETE /identity/v3/projects/53eefbdc480243dd9234471bd8fe9d87 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 12:45:41.541245 dv-cinder-860244-1-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=62 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 12:45:42.138003 dv-cinder-860244-1-iscsi devstack@keystone.service[93092]: DEBUG dbcounter [-] [93092] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}